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SUMMARY  OF  VOLUME  3 

TECHNICAL  PROBLEM 

he  large  network  research  conducted  by  Network  Analysis 
Corporation  under  contract  with  the  Advanced  Research  Projects 
Agency  had  the  following  objectives: 

. To  define  delay  and  reliability  performance  criteria 
for  large  network  design. 

. To  identify  adequate  network  structures  for  large, 
distributed  requirements. 

. To  investigate  the  impact  of  system  parameters  and 
characteristics  (e.g.,  hardware  cost,  line  cost, 
traffic  volume  etc.)  on  large  network  design  and 
performance. 

. To  determine  the  impact  of  satellite  carriers  on 
large  network  design. 

. To  determine  the  impact  of  precedence  and  priority 
procedures  on  network  design  and  performance  in  a 
large  network  environment,  with  a wide  variety  of 
traffic  types  and  delay  requirements.  _ 

GENERAL  METHODOLOGY 


To  achieve  the  research  objectives  the  following  methodology 
was  used: 
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Review  of  traditional  structures  for  small  and 
medium  scale  networks,  to  determine  their  appli- 
cability to  large  network  implementations. 

Definition  of  a gamut  of  alternative  large  net- 
work architectures.  , 

Development  of  an  integrated  computer  package  for 
the  design  and  performance  evaluation  of  large 
networks,  combining  traditional  design  techniques 
and  newly  developed  techniques. 

Definition  of  two  large  network  examples  (with 
approximately  1,000  and  10,000  subscribers  re- 
spectively) constructed  in  conjunction  with  ARPA, 
and  reflecting  the  Department  of  Defense  require- 
ments. 


Application  of  the  large  network  techniques  to 
the  above  examples,  to  obtain  the  desired  cost 
and  performance  tradeoffs. 


TECHNICAL  RESULTS 

The  major  accomplishments  presented  in  the  report  are  summarized 

An  integrated,  multilevel  network  architecture  was 
shown  to  be  the  most  cost-effective  solution  for 
large,  distributed  requirements.  The  multilevel 
architecture  offers  the  best  trade  off  between 
line  costs,  switch  costs  and  delay  and  reliability 
performance.  Furthermore,  it  shows  excellent  con- 
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The  sensitivity  of  the  optimal  solution  to  changes 
in  system  parameters  was  extensively  investigated 
for  a two- level  network  structure  consisting  of  a 
distributed  backbone  network  and  star  local  access 
subnetworks.  In  particular,  it  was  shown  that  the 
optimal  number  of  switches  is  most  sensitive  to 
variations  in  local  access  unit  mileage  line  cost 
and  switch  cost,  and  least  sensitive  to  variations 
in  backbone  unit  mileage  line  cost.  Similarly,  it 
was  shown  that  total  cost  is  most  sensitive  to 
variations  in  local  access  cost,  less  sensitive 
to  variations  in  backbone  cost,  and  least  sensitive 
to  variations  in  switch  cost. 

It  was  shown  that  for  large  nationwide  requirements 
satellite  communications  using  dedicated  earth  stations 
become  more  cost-effective  than  terrestrial  communi- 
cations only  if  the  number  of  stations  required  is 
relatively  small.  In  particular,  it  was  shown  that 
for  a network  environment  with  1,000  subscribers 
a pure  satellite  network  with  no  terrestrial  tails 
and  with  approximately  300  earth  stations  is  50% 
more  expensive  than  a pure  terrestrial  implementation; 
while  a hybrid  satellite-terrestrial  implementation 
using  an  8 node  satellite  backbone  provides  10% 
savings  with  respect  to  the  terrestrial  implementation. 


A two-level  trunk  priority  was  found  to  be  the  most 
cost-effective  in  a Defense  Communication  environment 
in  that  it  provides  all  of  the  benefits  that  can  be 
obtained  from  trunk  priorities,  at  minimum  imple- 
mentation cost.  Among  the  performance  improvements 
with  respect  to  the  no  priority  case  are;  50%  delay 
reduction  for  high  priority  packets,  20%  increase 
in  effective  end-to-end  bandwidth,  20%  S/F  buffer 
saving,  and  nonblocking  network  access  for  high 
priority  traffic. 
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DEPARTMENT  OF  DEFENSE  IMPLICATIONS 


The  DOD  is  presently  planning  the  integration  of  its  data 
communication  requirements  in  a highly  efficient,  cost-effective 
and  reliable  data  network.  In  the  short  term,  some  of  the  results 
and  tradeoffs  developed  in  this  study  may  prove  useful  to  DOD  for 
the  initial  planning  of  the  integrated  network.  In  the  long  term, 
the  study  provides  guidance  for  network  expansions  and  reconfigura- 
tions required  by  future  changes  in  the  network  environment  (e.g., 
traffic  growth,  change  of  traffic,  change  of  performance  require- 
ments , etc . ) . 


IMPLICATIONS  FOR  FURTHER  RESEARCH 

> The  major  areas  identified  for  further  research  include: 

• Development  of  cost-performance  tradeoffs  for 
large  networks  with  both  voice  and  data  traffic, 
and  with  circuit  switched  (or  hybrid  packet  and 
circuit  switched)  backbone  implementations. 

• Development  of  cost-performance  tradeoffs  for 
mixed  terrestrial  and  satellite  implementations, 
as  a function  of  different  channel  access  schemes, 
routing  disciplines  and  flow  control  procedures. 

• Study  of  the  interaction  between  trunk  priorities, 
network  acceptance  priorities,  and  flow  control 
procedures,  and  of  criteria  for  the  selection 

of  the  most  effective  blend  between  link  controls, 
end-to-end  controls  and  congestion  controls  to 
meet  given  performance  requirements  in  a large 
network  environment. 
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1.  LARGE  SCALE  NETWORK  DESIGN;  PROBLEM  DEFINITION  AND  SUMMARY 

OF  RESULTS 

1 . 1 GENERAL 


The  definition  of  large  data  networks  is  generally  based  on 
the  following  criteria: 

1.  Number  of  Host  computers  and  terminals, 


2.  Number  of  geographically  distinct  installations. 


3.  Geographical  spread  of  Host  and  terminal  requirements,  and 


4.  Traffic  volume  and  characteristics. 


In  this  study,  we  refer  to  a large  network  as  a network  with 
on  the  order  of  one  hundred  or  more  distinct  installations  (each  with 
one  or  more  terminals  and/or  Hosts)  distributed  nationwide  and  with 
traffic  volume  and  characteristics  that  require  the  use  of  high  speed 
backbone  trunks  (say  50  Kbps  or  more) . 

Data  networks  are  often  classified  on  the  basis  of  traffic 
pattern.  We  distinguish  betweem  centralized  networks  in  which  all 
traffic  requirements  are  to  and  from  a central  nodel  and  distributed 
networks  in  which  traffic  requirements  are  widely  distributed  among 
different  node  pairs  (terminal-computer  and  computer-computer) . The 
emphasis  of  this  study  is  on  distributed  networks,  although  most  of 
the  concepts  apply  also  to  centralized  networks  as  a special  case. 

Finally,  we  often  distinguish  between  computer-oriented  data 
networks  in  which  inter-computer  communications  are  predominant,  and 
terminal-oriented  data  networks -with  mostly  terminal-to-computer 
communications.  Again,  the  concepts  presented  here  are  valid  for 
both  types  of  networks.  However,  the  full  range  of  alternatives 
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that  are  considered  here  applies  only  to  networks  with  a predominant 
terminal  traffic  component. 

In  this  study,  therefore,  we  are  concerned  primarily  with  large, 
distributed,  terminal-oriented  networks.  Networks  of  this  kind 
generally  result  from  the  integration  of  several  centralized  and 
distributed  subnetworks  for  the  purpose  of  achieving  communications 
line  economies  and  inter-network  communications  capabilities. 

Important  examples  of  this  integration  trend  in  the  United  States  can 
be  found  in  large  corporate  networks  and  in  the  proposed  AUTODIN  II 
Defense  Communications  network. 

The  most  cost-effective  structure  for  large,  distributed, 
terminal-oriented  networks  is  a two-level  hierarchical  structure 
consisting  of  a backbone  network  (high  level)  and  local  access  net- 
works (low  level).  The  backbone  network  is  characterized  by  dis- 
tributed traffic  requirements  and  is  generally  implemented  using  the 
packet-switching  technology.  Local  access  networks,  on  the  other 
hand,  have,  in  general,  a centralized  traffic  pattern  (all  the 
traffic  is  to  and  from  the  gateway  backbone  node)  and  are,  therefore, 
implemented  with  centralized  techniques  such  as  multiplexing,  con- 
centration, and  polling. 

The  purpose  of  this  study  is  to  identify  alternative  strategies 
and  cost-performance  tradeoffs  for  the  design  of  large  two-level 
hierarchical  networks.  In  particular,  several  techniques  for  terminal 
access  are  evaluated,  and  the  impact  of  number  and  location  of  back- 
bone nodes  on  network  cost  and  performance  is  thoroughly  investigated. 
Traditional  network  analysis  and  design  tools  are  modified  and  extended, 
and  are  applied  to  the  two-level  network  study. 

Large  network  alternatives  are  then  evaluated  for  the  proposed 
AUTODIN  II  data  network,  which  will  integrate  the  nationwide  Defense 
Communications  requirements  in  a cost-effective  and  reliable  hier- 
archical structure.  For  this  application,  detailed  network  designs 
are  carried  out  for  various  strategies,  and  multidimensional  trade- 
offs are  derived. 
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The  present  chapter  serves  as  introduction  to  the  large  network 
study,  and  has  the  purpose  of  defining  the  scope  of  the  problem. 
Discussed  in  this  chapter  are: 


The  formulation  of  the  large  network  design  problem;  and 


4.  The  Defense  Communications  requirements 


1.2  PREVIOUS  CONTRIBUTIONS 


Considerable  research  effort  was  dedicated  in  the  past, 
under  previous  ARPA  contracts,  to  the  design  of  large  data  networks 
The  contributions  can  be  classified  into  two  categories: 


The  major  results  and  achievements  of  the  research  under 
Category  A are  briefly  summarized  in  the  following  points: 


1.  For  a very  large,  computer-oriented  network,  a hierarchical 
implementation  was  shown  to  be  more  cost-effective  than  a 
homogeneous  implementation  (lower  delay,  better  line  economies, 
etc.)  [NAC,  1974]. 
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2.  The  feasibility  of  efficient  routing  and  flow  control 
techniques  for  large,  hierarchical  networks  was  demonstrated 
[NAC,  1973], 

3.  An  efficient  algorithm  (the  cut-saturation  algorithm) 
was  developed  for  the  topological  design  of  distributed 
packet  networks  [NAC,  1973)  [NAC,  1974). 

4.  The  interactive  graphics  capability  was  developed  for 
the  topological  design  program  to  enable  efficient  man-machine 
interaction  in  the  heuristic  solution  of  large,  complex  network 
problems  [NAC,  1974). 
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5.  The  impact  of  satellite  links  on  a distributed  packet 
network  architecture  was  investigated,  and  guidelines  for 
the  installation  of  ground  stations  were  provided. 

The  research  on  terminal  access  networks  has  proceeded  in 
parallel  with  the  research  on  computer  networks.  The  most  significant 
results  are  summarized  here: 

1.  An  efficient  algorithm  was  developed  for  the 
topological  layout  of  multidrop  lines  connecting 
terminals  to  a central  facility  (Host  computer, 
concentrator,  or  backbone  node)  [NAC,  1972). 


2.  An  efficient  algorithm  was  developed  for  the 
regional  partitioning  of  terminals  and  the  selection 
of  regional  concentrator  locations  in  a large, 
centralized  network  [NAC,  1974). 


3*  A*  • 


Network  Analysis  Corporation 


The  above  results  indicate  that  the  research  has  proceeded,  in 
the  past,  along  two  independent  directions: 


The  two  efforts  are  now  naturally  converging  to  the  same  objective: 
the  design  of  large,  distributed,  terminal-oriented  networks.  The 
present  study  will  capitalize  on  the  distributed  computer  network 
results  for  the  design  of  the  backbone  network,  and  on  the  terminal 
access  results  for  the  design  of  local  distribution  subnetworks. 

The  new,  challanging  problems  that  must  be  solved  in  the 
present  study  are  the  following: 


Selection  of  backbone  node  number  and  locations,  and 


2.  Assignment  of  terminals  and  Hosts  to  backbone  nodes 
(partitioning  problem) . 


It  is  shown,  in  fact,  in  Section  6 that  if  backbone  node  locations 
and  partitions  are  assigned,  the  global  design  problem  is  reduced 
to  two  separate  subproblems;  namely,  backbone  design  and  local  access 
design,  which  can  be  solved  with  existing  methods. 

The  backbone  location  problem  is  somewhat  related  to  the 
regional  concentrator  location  problem  already  considered  and 
solved  within  the  terminal  access  research  effort.  Therefore, 
the  experience  and  some  of  the  results  obtained  there  can  also  be 
applied  here.  However,  the  cost  tradeoffs  between  backbone  trunks 
and  local  access  lines  in  a large,  distributed  net  are  very  different 
from  the  tradeoffs  between  concentrator-to-Host  trunks  and  local 
access  lines  in  a centralized  network.  A considerable  research  effort 
to  study  the  new  tradeoffs  is  therefore  anticipated. 
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1.3  ALTERNATIVE  STRATEGIES  FOR  BACKBONE  NETWORK  IMPLEMENTATION 


In  this  study,  we  assume  that  the  backbone  network  is  operated 
using  the  packet-switched  technology.  Packet  switching  is,  in  fact, 
a simple,  reliable,  and  cost-effective  solution  for  distributed  data 
communications  requirement  consisting  of  a mix  of  interactive  and 
bulk  data  traffic. 

If  the  traffic  is  predominantly  represented  by  file  transfers 
and  digitized  speech,  then  a hybrid  combination  of  packet  and  circuit 
switching  may  be  more  effective  than  simple  packet  switching.  However, 
the  issue  of  feasibility  and  effectiveness  of  a hybrid  implementation 
and,  more  generally,  the  comparison  of  circuit  versus  packet-switched 
disciplines  is  a formidable  research  topic  on  its  own  right  and  is 
beyond  the  scope  of  this  study. 

Within  the  packet-switched  concept,  several  architectural 
alternatives  are  available  and  must  be  considered  for  a cost-effective 
backbone  design. 


1.3.1 


Line  Alternatives 


The  main  issue  here  is  whether  to  use  only  terrestrial  links 
or  a combination  of  terrestrial  and  satellite  links.  In  the  latter 
case,  the  satellite  channel  access  technique  must  also  be  selected 
from  several  possible  alternatives  (point-to-point,  TDMA,  FDMA, 
slotted  ALOHA,  etc.). 

Terrestrial  links  are  available  under  several  different  service 
offerings,  e.g. , Telpak,  DDS  (Digital  Dataphone  Service),  Wideband 
Service  8000,  and  Tl  carrier.  For  a given  connection,  the  most 
cost-effective  arrangement  among  the  available  services  must  be 
selected. 

In  general,  the  selection  of  the  best  communications  carrier 
is  not  based  uniquely  on  line  cost,  but  on  channel  delay,  reliability, 
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and  error  quality  considerations.  Furthermore,  the  use  of  some 
carriers  (e.g.,  satellite  and  Tl  carriers)  may  require  the  development 
of  specialized  software  and  hardware  interface  capabilities  in  the 
nodal  processors.  In  this  case,  the  additional  nodal  cost  must  also 
be  considered. 

Finally,  the  selection  of  the  carrier  may  have  an  impact  on 
the  selection  of  backbone  node  locations.  For  example,  if  DDS 
service  is  used,  backbone  nodes  will  preferably  be  installed  in 
digital  cities.  Similarly,  if  domestic  satellite  service  is  used, 
backbone  nodes  will  preferably  be  installed  near  carrier  ground 
stations. 

1.3.2  Node  Alternatives 

A conventional  packet  switch  (i.e.  , ARPANET  IMP)  has  some 
well-defined  limits  in  terms  of  throughput  and  number  of  modem  inter- 
faces it  can  support.  In  a large  network  implementation,  nodal 
throughput  and  interface  requirements  often  exceed  such  limits  and, 
therefore,  make  it  necessary  to  develop  new  nodal  architectures  with 
upgraded  capacity. 

One  possible  solution  consists  of  combining  several  conventional 
nodes  in  a fully  interconnected  cluster.  The  cluster  can  thus  be 
viewed  as  a supernode  with  an  external  throughput  and  modem  interface 
capacity  higher  than  the  single  IMP  capacity,  but  obviously  lower  than 
the  sum  of  the  capacities  of  all  the  individual  IMP'S  in  the  cluster 
since  some  of  the  interfaces  are  used  for  internal  connections,  and 
transit  traffic  typically  traverses  two  IMP'S  in  the  cluster. 

Another  approach  to  higher  nodal  capacity  is  the  multiprocessor 
switch  implementation  (e.g.,  Pluribus  IMP).  The  multiprocessor  switch 
is  a device  with  several  processor,  I/O,  and  memory  modules  inter- 
connected together  and  performing  different  packet  switch  functions  in 
parallel.  The  multiprocessing  environment  and  the  overlap  of  different 
functions  lead  to  higher  throughput  and  modem  interface  capability. 
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The  multiprocessor  switch  solution  is,  in  general,  more 
systematic  and  powerful  than  the  cluster  solution.  However,  the 
cluster  solution  may  be  more  attractive  in  some  applications  for  the 
following  reasons: 

1.  No  additional  software  development  cost  (as 
opposed  to  the  cost  for  the  development  of  new, 
multiprocessor  switch  software). 

2.  Easier  integration  in  a network  with  con- 
ventional nodes  (e.g. , each  node  in  the  cluster 
can  be  reloaded  from  other  network  nodes,  etc.). 

3.  Better  redundancy. 

1.3.3  Topological  Structure  Alternatives 

For  a small  or  medium  size  backbone  network  (say  up  to  20  or 
30  nodes) , the  typical  structure  is  a homogeneous  structure  in  which 
all  nodes  have  identical  hardware  and  software  and  perform  identical 
functions.  For  networks  of  a larger  size,  a two-level  hierarchical 
structure  may  result  to  be  more  cost-effective  [NAC,  1972]  and  should 
be  considered  as  a possible  alternative. 

In  a two- level  packet  network,  the  high  level  net  has  higher 
node  and  link  capacities  than  the  low  level  "subnets."  A subnet 
is  a packet  network  connected  to  the  high  level  net  through  two  or 
more  high  level  nodes  (which  act  as  gateways) . Communications  between 
different  subnets  are  performed,  in  general,  via  the  high  level  net. 
However,  neighbor  subnets  may  communicate  with  each  other  directly 
if  routing  protocols  permit. 
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The  objectives  of  a two-level  packet  network  implementation  are 
the  following: 

1.  Use  of  high  bandwidth,  cost-effective  communications 
°fferings  (e.g.,  satellite  channel  or  T1  carrier)  for  long 
distance  and  high  capacity  trunks  between  high  level 
switches,  and 

2.  Reduction  of  local  access  costs  from  terminals  to  backbone 
switches  obtained  by  providing  a large  number  of  geographically 
well-distributed  low  level  switches. 
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1.4  ALTERNATIVE  STRATEGIES  FOR  LOCAL  DISTRIBUTION 

The  connection  of  Host  computers  and  terminal  to  backbone  nodes 
can  be  established  using  a variety  of  different  techniques.  In  our 
study,  we  restrict  our  consideration  to  the  alternatives  discussed 
below. 

1.4.1  Host  Access 

Direct,  dedicated  connection  from  the  Host  to  backbone  node 
is  assumed.  Line  speed  is  selected  so  as  to  satisfy  Host  traffic 
requirements  with  an  acceptable  transmission  and  queueing  delay.  If  a 
single  line  connection  does  not  provide  the  required  reliability,  full 
line  backup  or  dual  homing  is  considered. 

In  practical  applications,  two  or  more  colocated  Hosts  may  be 
connected  to  a common  Front  End  Processor  (FEP)  which,  in  turn,  is 
connected  to  the  backbone  node  via  dedicated  line.  This  case  clearly 
reduces  to  the  previous  one  by  simply  merging  the  colocated  Hosts  into 
a Host  cluster. 

1.4.2  ~ Terminal  Access 

The  following  techniques  are  considered  for  the  connection  of 
terminals  to  backbone  nodes: 

1.  Dialup, 

2.  Dedicated  circuit  (point-to-point) , 

3.  Polling  over  a multidrop  circuit, 
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4.  Frequency  Division  Multiplexing  (FDMX)  over  a 
multidrop  circuit, 

5.  TDMX  or  FDMX  over  a point-to-point  dedicated  circuit, 

6.  Clustering  of  colocated,  identical  terminals  via  a 
Terminal  Control  Unit,  and 

7.  Concentration  of  local  and  remote  terminals  via  a con- 
centrator. 

I 

A local  access  network  can  consist  of  several  different  hier- 
archical levels  (e.g.,  FDMX  at  the  lowest  level,  then  TDMX,  then 
concentration,  and  finally,  polling  at  the  highest  level) . In 
principle,  therefore,  there  exists  a very  large  gamut  of  possible 
access  structures.  In  this  study,  we  limit  our  considerations  to 
two-level  local  access  structures  in  which  the  low  level  can  be 
implemented  with  any  access  technique  from  1 through  6 and  the  high 
level  (if  present)  is  implemented  with  technique  7.  (This  is  indeed 
no  loss  of  generality  since  a great  majority  of  applications  obey 
this  rule. ) 
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For  the  low  level  local  access  implementation,  if  all  the 
terminals  are  identical  or  have  similar  line  protocols  and  traffic 
characteristics,  a unique  access  technique  can  be  used  (homogeneous 
local  distribution  network) . 

If,  on  the  other  hand,  terminals  differ  widely  from  one  another 
(e.g. , teletypes,  CRT's,  RJE's,  intelligent  terminals,  etc.),  the 
best  strategy  will  consist,  in  general,  of  a combination  of  different 
access  techniques,  one  for  each  subclass  of  terminals  (heterogeneous 
local  distribution  network) . 

The  results  of  this  study  apply  both  to  homogeneous  and  hetero- 
geneous networks.  In  particular,  the  heterogeneous  strategy  consisting 
of  a combination  of  techniques  2,  5,  and  6 will  be  considered  for  the 
Defense  Communications  application. 

At  a higher  level,  the  access  to  the  backbone  net  is  provided  by 
concentrators.  In  this  study  we  assume  that  concentrators  are  mini- 
Hosts  equipped  with  all  the  network  protocols  of  a regular  Host,  like 
the  ELF  or  ANTS  in  ARPANET.  The  connections  from  concentrator  to 
backbone  are  therefore  implemented  with  the  same  criteria  as  the  Host- 
to-backbone  connections. 

1.4.3  Segregation  vs  Integration 

It  may  be  observed  that  in  some  applications  there  are  subsets 
of  terminals  that  communicate  predominantly  with  only  one  Host.  In 
this  case,  the  most  appropriate  local  distribution  strategy  may  cor- 
respond to  the  direct  connection  of  the  terminals  to  the  Host,  rather 
than  to  the  backbone  node.  In  the  limit,  if  the  large  terminal  and 
Host  population  can  be  partitioned  into  several  disjoined  subsystems 
and  communications  requirements  exist  only  within  each  subsystem  with 
no  intercommunications  between  different  subsystems,  it  may  result  that 
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the  best  strategy  is  a segregated  strategy  (each  subsystem  is  connected 

via  a private  network)  rather  than  an  integrated  strategy  (all  sub- 

« 

systems  share  a common  backbone  network) . 

The  emphasis  of  this  research  is  on  integrated  strategies. 
Therefore,  most  f the  tradeoff  studies  and  design  guidelines  are 
based  on  the  assumption  that  all  Host-to-Host  and  terminal-to-Host 
traffic  is  switched  through  the  backbone  network. 
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1.5  PERFORMANCE  MEASURES 

Given  the  set  of  network  requirements  (number  and  location  of 
Hosts  and  terminals,  traffic  pattern,  traffic  characteristics,  etc.), 
the  performance  of  the  network  will  depend,  in  part,  on  the  hardware 
selection  and  architectural  design  (e.g.,  line  layout,  buffer  size, 
line  speed,  processor  speed,  etc.)  and,  in  part,  on  the  communications 
software  design  (line  protocols,  routing  protocols,  end-to-end  protocols, 
etc . ) . 

Some  of  the  performance  measures  will  depend  more  critically  on 
hardware  selection  and  architectural  design  (e.g.,  path  reliability) 
while  others  will  be  more  directly  related  to  the  communications  soft- 
ware development  (e.g.,  flow  control,  congestion  protection,  stability 
etc. ) 

Since  the  main  purpose  of  this  study  is  the  design  (rather  than 
the  control)  of  large  network  architectures,  we  identify  here  the 
performance  measures  which  are  directly  related  to  network  architecture 
and  must,  therefore,  be  considered  during  the  design  phase. 

1.5.1  Delay 


The  delay  from  terminal  to  Host  in  a hierarchical  network  struc- 
ture is  given  by  the  sum  of  the  delays  in  the  local  access  segments 
(terminal-to-backbone  and  Host-to-backbone)  plus  the  delay  over  the 
backbone  segment.  Since  delay  is,  in  general,  a random  variable, 
average  values  may  be  considered.  Network  delay  may  be  defined  there- 
fore as  the  maximum  of  the  average  (over  time)  delays  evaluated  between 
all  node  pairs. 

This  definition,  however,  leads  to  a cumbersome  procedure  for 
the  evaluation  of  the  delay  requiring  the  examination  of  all  terminal- 
to-Host  pairs.  Furthermore,  the  constraint  on  maximum  delay  is  dif- 
ficult to  account  for  in  the  topological  optimization  problem. 
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For  design  purposes , it  is  more  effective  to  introduce  the 
following  (approximate)  definition  of  network  delay  T: 

T = MAX  (T  (TB) ) + AVG  (T  (BB) ) + MAX  (T  (BH)  ) (1) 

where : 

MAX (T (TB) ) is  the  maximum  over  all  terminal-to-backbone  node 
pair  delays. 

AVG (T (BB) ) is  the  average  delay  in  the  backbone  network  (the 
average  is  over  time  and  over  all  node  pairs) . 

MAX (T (BH) ) is  the  maximum  over  all  backbone  node-to-Host  pair 
delays. 


i 


Delays  (unless  otherwise  specified)  are  evaluated  in  peak  hour  traffic 
conditions  and  for  the  average  block  length  of  a typical  transaction. 

Average  rather  than  maximum  delay  is  used  for  the  backbone  segment 
since  the  former  is  much  easier  to  handle  for  design  and  evaluation 
purposes  in  a distributed,  packet-switched  network.  The  approximation 
is  quite  acceptable  in  practical  applications  since  backbone  delays 
are  generally  much  lower  than  local  access  delays  and  thus  have  only 
marginal  impact  on  global  delay  performance  T. 

The  above  delay  definition  was  introduced  under  the  assumption 

that: 


1.  All  the  terminals  are  similar  and  are  dedicated  to  the 
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* In  practicality,  a large  network  may  support  several  different  types 

I « 

of  terminals  and  applications.  Furthermore,  a given  terminal  may  com- 
municate with  only  a limited  number  of  Hosts.  Finally,  there  may  also 
exist  Host— to— Host  communications. 

To  handle  the  more  general  case,  we  need  to  consider  a vector 
of  delay  values,  where  each  entry  corresponds  to  a given  type  of 
application.  For  example,  we  may  have  different  values  of  delay  for 

| j 

interactive  type  traffic,  RJE  transfers,  and  Host-to-Host  communications. 

The  existence  of  different  delay  requirements  for  different  ter- 
minal  type  and  application  has  a critical  impact  on  the  design  of  the 
local  access  subnetwork.  In  particular,  some  access  strategies  may 

not  be  feasible  for  a given  terminal  type  under  given  delay  require- 

f 

ments.  Backbone  network  design,  on  the  other  hand,  is  quite  insensitive 
to  the  presence  of  different  delay  requirements  in  the  local  subnet- 
works,  provided  that  AVG (T (BB) ) is  small  as  compared  to  any  such  re- 
quirements . 

1.5.2  Throughput 


Throughput  capability  of  a network  may  be  defined  and  evaluated 
according  to  the  procedure  described  in  the  sequel.  For  a given  pat- 
tern of  input  traffic  requirements,  input  rates  are  scaled  up  uniformly 
(so  that  their  ratios  are  kept  constant)  until  the  delay  (which  is  an 
increasing  value  of  input  rates)  reaches  the  maximum  admissible  value 
TMAX  specified  for  that  particular  network.  The  sum  of  the  input  rates 
that  yield  T = TMAX  is  defined  to  be  the  throughput  for  the  network 
under  a specified  traffic  pattern. 

Throughput  performance,  therefore,  is  a function  not  only  of  net- 
work topology,  but  also  of  traffic  pattern  and  delay  requirements. 

Since  traffic  pattern  knowledge  is  often  not  accurate  in  large  data 
network  applications,  throughput  performance  must  be  associated  with 
the  sensitivity  of  network  throughput  to  variations  of  traffic  patterns. 
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1.5.3  Reliability 

In  a network  with  less  than  perfectly  reliable  nodes  and  links, 
the  reliability  of  communications  between  a terminal  and  a Host 
computer  is  defined  as  the  probability  of  the  existence  of  at  least 
one  path  connecting  them.  Network  reliability  may  be  defined  as  the 
minimum  of  all  node  pair  reliabilities  (i.e.,  the  reliability  between 
the  most  unfavorable  node  pair)  or  as  the  average  over  all  node  pairs. 
In  reality,  both  minimum  and  average  (and  possibly  distribution  vs. 
fraction  of  node  pairs)  are  important  to  fully  characterize  network 
reliability  performance.  In  this  study,  the  minimum  is  chosen  as 
measure  of  reliability. 

In  a two-level  hierarchical  network,  the  reliability  r for  a 
given  terminal-to-Host  pair  is  calculated  as  follows: 

r = r (TB)  x r (BB)  x r (BH) 

where  r (TB) , r (BB)  and  r (BH)  are  the  reliability  from  terminal-to- 
backbone  node,  between  backbone  nodes,  and  from  backbone  node  to  Host, 
respectively.  Network  reliability  R is  therefore  defined  as  follows: 

R = minimum  (over  all  pairs)  {r (TB)  x r (BB)  x r (BH) } 


Clearly: 

R _>  R'  = Min  r (TB)  x Min  r (BB)  x Min  r (BH) 

R' , therefore,  is  a conservative  estimate  of  R.  It  is  also  easier 
to  evaluate,  since  it  allows  the  independent  reliability  analysis  of 
backbone  and  local  access  subnets.  In  our  study,  we  use  definition  R' . 

The  definition  of  network  reliability  can  be  extended  to  handle 
problems  with  different  subsystems  having  different  reliability  re- 
quirements (in  which  case,  a different  value  is  calculated  for  each 
subsystem) . 

Extensions  to  the  basic  definition  also  allow  analysis  of  dual 
homing  strategies,  where  a Host  or  terminal  is  connected  to  two 
backbone  nodes. 
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1.6  PROBLEM  FORMULATION 

The  topological  design  problem  for  a two-level  hierarchical 
network  can  be  formulated  as  follows: 

Given : 

1.  Terminal  and  Host  locations, 

2.  Matrix  of  traffic  requirements  (terminal-to-Host 
and  Host-to-Host) , 

3.  Delay  requirements  (possibly  different  requirements 
for  different  subsystems) , 

4.  Reliability  requirements  (possibly  different  re- 
quirements for  different  subsystems) * 

5.  Candidate  sites  for  backbone  nodes,  and 

6.  Cost  elements  (line  tariff  structures,  nodal  pro- 
cessor costs,  hardware  costs,  etc.). 

Minimize: 

Total  communications  cost  D: 

D = (backbone  line  costs)  + (backbone  node  costs) 

+ (local  access  line  costs)  + (local  access 
hardware  costs) 


Such  that 


The  global  design  problem  consists  of  two  separate  subproblems 


The  two  subproblems  interact  with  each  other  through  the  following 
parameters: 


Backbone  node  number  and  locations 


2.  Terminal  and  Host  association  to  backbone  nodes 
(partitioning) , 


3.  Delay  requirement  for  backbone  network,  and 


Once  the  above  variables  are  specified,  the  two  subproblems  can  be 
solved  independently. 

The  problem  structure  typically  leads  to  heuristic  solutions 
of  iterative  nature.  An  example  of  iterative  approach  is  the  following 


1.  Each  subproblem  is  optimized  after  assigning  reasonable 
heuristic  values  to  parameters  1 through  4 , 
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2.  The  parameters  are  modified  in  the  direction  that  yields 
maximum  cost  savings, 

3.  The  optimization  of  the  subproblems  is  carried  out  under 
the  new  parameter  assignment, 

4.  If  local  minimum  is  reached,  stop.  Otherwise,  go  to  2. 

Several  different  local  minima  can  be  reached  by  using  different 
starting  assignments  of  the  parameters. 

In  large  network  applications,  the  global  design  is  generally 
rather  insensitive  to  the  selection  of  backbone  delay  and  reliability 
requirements.  In  fact,  the  high  volume  and  distributed  pattern  of 
traffic  demands  in  the  backbone  network  generally  require  high  back- 
bone trunk  capacity  (with  transmission  delay  substantially  lower  than 
the  delay  on  local  distribution  lines) , high  backbone  connectivity  and 
high  nodal  redundancy.  As  a consequence,  delay  and  reliability  in  the 
backbone  net  are  typically  one  order  of  magnitude  better  than  in  local 
distribution  nets.  Therefore,  variations  of  backbone  delay  and  relia- 
bility requirements  have  little  effect  on  global  performance. 

The  new  variables  that  characterize  large  network  design  and  dis- 
tinguish it  from  traditional  network  problems  are  therefore  partitioning 
and  backbone  node  location.  One  of  the  objectives  of  this  study  will 
be  to  determine  the  sensitivity  of  global  cost  and  performance  to  vari- 
ations of  such  parameters. 
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1.7  CASE  STUDY:  INTEGRATED  DEFENSE  COMMUNICATIONS  NETWORK 

The  data  communications  requirements  within  DOD  arise  from  a 
variety  of  different  applications.  Presently,  a separate  communica- 
tions network  is  dedicated  to  each  application,  and  the  system  con- 
sists of  the  superposition  of  several  different  networks  with  typi- 
cally cannot  communicate  with  each  other. 

In  order  to  obtain  better  line  and  hardware  economies,  and  a more 
flexible  internet  communications  capability,  the  DOD  is  planning  to 
integrate  most  of  the  data  transmission  requirements  on  AUTODIN  II, 
a new  data  network  implementing  the  packet-switching  technology. 

Among  the  important  decisions  required  during  the  network  planning 
process  are: 

1.  Number,  location  and  structure  of  the  backbone  nodes. 

2.  Carrier  offerings  to  be  used  for  the  implementation  of 

backbone  trunks  (terrestrial  and/or  satellite) • 

3.  Local  access  techniques. 


The  above  requirements  indicate  that  AUTODIN  II  is  a very  repre- 
sentative example  of  a large,  distributed  data  network,  which  can  be 
used  as  a case  study  in  the  present  investigation.  Therefore,  design 
criteria  and  cost-performance  tradeoffs  developed  in  general  network 
models  will  be  applied  to  (and  verified  on)  the  AUTODIN  II  require- 
ments. 

A summary  of  Defense  Communications  requirements  and  character- 
istics considered  in  the  study  is  presented  below. 
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A.  AUTODIN  II  will  consist  initially  of  35  systems,  of 
which  3 are  presently  implemented  with  distributed  computer-to- 
computer  networks  (AUTODIN  I,  WWMMCS  and  ALS)  and  32  are  imple- 
mented with  centralized,  terminal -oriented  networks. 

B.  The  35  systems  include  87  Host  computers  and  over  1,000 
terminals,  nationwide  distributed  and  installed  at  over  250 
distinct  locations. 

C.  Total  net  input  traffic  (sum  of  all  terminal  and  Host 
requirements)  is  estimated  to  be  1,250  Kbps. 

D.  Delay  requirements  vary  according  to  the  application.  In 
this  study,  delay  requirements  are  assumed  to  be  less  or  equal 
to  one  second  for  Host-to-Host  communications,  and  less  or 
equal  to  two  seconds  for  terminal-to-Host  communications. 

E.  Reliability  requirements  vary  according  to  the  system. 
Systems  are  classified  into  two  categories:  critical  systems, 
that  must  be  up  more  than  99.95  percent  of  the  time;  and 

non  critical  systems,  that  must  be  up  more  than  99  percent 
of  the  time. 

F.  Security  of  communications  is  required  for  12  of  the  35 
systems.  More  precisely,  49  out  of  the  total  87  computers, 
and  66  out  of  the  1000  plus  terminals  are  secure.  Communica- 
tions between  such  computers  and  terminals  are  made  secure  by 
introducing  link  and/or  end-to-end  encryption  devices. 

For  the  Defense  Communications  requirements,  the  following 
‘studies  will  be  performed: 
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A.  Cost-evaluation  of  various  segregated  network  strategies, 
assigning  a private  network  to  each  system. 

B.  Cost-evaluation  of  various  integrated  strategies,  using 
different  approaches  for  backbone  network  design. 

C.  Sensitivity  of  global  cost  and  performance  to  changes  in 
backbone  node  number  and  location. 

D.  Cost-effectiveness  of  satellite  links  for  backbone  communi- 
cations. 

E.  Integration  of  findings  in  A,  B,  C,  and  D to  determine  multi- 
dimensional tradeoffs  for  the  Defense  Communications  require- 


ments. 
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1 . 8 COST  TRENDS  OF  THE  BACKBONE  SWITCH  LOCATION  SOLUTION 
1.8.1  System  Models 

The  backbone  switch  location  procedure  developed  in  the  course 
of  this  study  was  applied  to  the  following  models: 

1.  AUTODIN  II:  This  is  the  integrated  communication 

network  to  include  the  ADP  systems  administered  by  the 
DOD.  The  AUTODIN  II  requirements  are  as  follows: 


Total  number  of  user  locations: 
number  of  host  computers: 
number  of  concentrators: 
number  of  TDMX's: 
number  of  TCU ' s : 
number  of  isolated  terminals: 
Total  traffic: 


1.26  Megabits/sec. 


A Randomly  Generated  Communication  (RGC)  System: 


This  system  was  generated  specifically  to  study  the 
impact  of  a tenfold  growth  of  AUTODIN  II  traffic  re- 
quirements on  the  switch  location  strategy.  Ter- 
minal locations  and  traffic  volume  are  generated  in 
proportion  to  population  distribution  of  the  United 
States.  The  traffic  and  location  pattern  of  this 
system  is  thus  in  general  more  uniform  than  that  of 
the  AUTODIN  II  system.  The  RGC  system  size  is  as 
follows: 
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Total 

number 

of 

user  locations: 

450 

number 

of 

host  computers: 

100 

number 

of 

concentrators : 

179 

number 

of 

TDMX ' s : 

45 

number 

of 

TCU ' s : 

45 

number 

of 

isolated  terminals: 

81 

Total 

traffic: 

12.0  Megabits/sec 

The  local  access  facilities  are  the  same  for  the  two  systems.  More 
powerful  switches  and  higher  speed  channels  are  required,  however, 
for  the  RGC  backbone  network,  since  the  traffic  volume  of  the  RGC 
system  is  approximately  10  times  larger  than  that  of  the  AUTODIN  II 
system. 


1.8.2  Trends  of  the  Optimal  Solution 

s 

The  most  important  trends  and  properties  discovered  during  our 
experiments  are  reported  below: 

A.  The  optimum  solution  is  in  the  range  of  9 to  12 
switches  for  both  AUTODIN  II  and  RGC  models  as  shown  by 
the  cost  per  Megabit  curves  as  a function  of  number  of 
switches  in  Figure  1.1.  It  must  be  coincidental  that 
in  our  study  the  optimum  number  of  backbone  nodes  for 
both  the  AUTODIN  II  system  and  the  RGC  system  fall 
within  the  same  range.  In  fact,  the  total  traffic  in 
the  RGC  system  is  10  times  higher  than  in  the  AUTODIN 
II  system,  but  the  capacity  of  the  backbone  switches 
and  lines  are  also  much  higher.  If  the  same  switch- 
ing and  line  facilities  were  employed  in  both  systems, 
we  would  expect  the  optimum  number  of  backbone  nodes 
for  the  RGC  system  to  be  much  higher  than  that  for  the 
AUTODIN  II  system. 

Topologies  for  AUTODIN  II  and  RGC  with  different 
switch  location  strategies  are  shown  in  Figures  1.2 
through  1.7. 
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FIGURE  1.1:  COST  PER  MEGABIT  FOR  AUTODIN  II  AND 

RGC  AS  A FUNCTION  OF  NUMBER  OF  BACKBONE 
SWITCHES 
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B.  The  curves  of  total  cost  versus  number  of  backbone 
nodes  are  very  flat.  In  the  range  considered  here  (5-20 
nodes) , the  difference  between  the  minimum  point  and  the 
maximum  point  usually  amounts  to  only  10  percent  of  the 
total  cost.  These  results  suggest  that  with  a modest 
computation  effort  we  can  obtain  fairly  good  designs 
with  total  cost  within  a few  percent  of  the  optimum. 

C.  The  cost  composition  of  the  AUTODIN  II  and  RGC 
topology  is  shown  in  Figures  1.8  and  1.9,  respectively. 
The  following  characteristics  are  observed: 


■ • 
h 


C.l  Of  the  three  main  components  of  the 
total  cost,  the  local  access  cost  is  clearly 
dominant  over  the  backbone  switch  node  cost 
and  the  backbone  line  cost.  This  fact  sug- 
gests that  in  trying  to  design  a near-optimum 
configuration,  we  should  spend  more  effort  in 
examining  the  local  cost  tradeoff  than  the 
other  two  components. 

C.2  When  the  number  of  backbone  nodes  in  the 
design  increases,  the  local  access  cost  usually 
decreases,  while  both  the  backbone  switch  cost 
and  the  backbone  line  cost  increase.  This  is 
intuitively  explained  by  the  fact  that  as  the 
number  of  nodes  in  the  backbone  increases: 

• The  terminals  are  closer  to  the 

backbone. 


■ l 
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FIGURE  1. 


FIGURE  1.9 


AUTODIN  II 


8:  COST  COMPOSITION  OF  THE  OPTIMAL  BACKBONE  DESIGN 

FOR  THE  AUTODIN  II  SYSTEMS 


RGC 


: COST  COMPOSITION  OF  THE  OPTIMAL  BACKBONE  DESIGN 

FOR  THE  RGC  SYSTEM 
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• The  local  traffic  decreases  (and 
thus  the  backbone  traffic  increases)  , 
because  terminals  and  parent  Hosts  are 
more  likely  to  be  assigned  to  different 
backbone  nodes,  and 

• The  number  of  links  in  the  back- 
bone increases. 

D.  As  for  the  impact  of  available  backbone  switch 
candidate  locations  on  network  cost,  the  designs  generated 
by  using  terminal  sites  as  backbone  candidates  have  lower 
overall  cost  than  the  designs  generated  by  using  the 
AUTOVON  locations  as  backbone  candidates.  This  can  be 
explained  as  follows: 

D.l  The  set  of  all  terminals  gives  a better 
choice  of  the  terminal  cluster  centers  than 
the  set  of  AUTOVON  locations,  because  of  a 
better  correlation  with  terminal  traffic  and 
geographical  distribution. 

D.2  The  set  of  all  terminals  contains  more 
elements  than  the  set  of  AUTOVON  locations, 
and  hence  offers  more  freedom  in  selecting 
the  geographical  location  of  switches. 

E.  In  studying  the  effect  of  unit  component  cost  vari- 
ations on  the  optimum  number  of  backbone  nodes  we  have 
observed  (see  Figures  1.10,  1.11,  and  1.12)  that  changes 
in  the  unit  cost  of  either  the  local  access  mileage  cost 
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NN:  OPTIMAL  NUMBER  OF  SWITCHES 


AUTODIN  II 


VOICE  GRADE  LINE  COST:  .83p  $/mile  x mo 

56  Kbps  LINE  COST:  5p  $/mile  x mo 

p:  MILEAGE  COST  SCALING  FACTOR 


FIGURE  1.10:  EFFECT  OF  LOCAL  ACCESS  MILEAGE  COST  VARIATIONS 


ON  OPTIMAL  NUMBER  OF  SWITCHES  FOR  AUTODIN  II  AND 


RGC  SYSTEMS  RESPECTIVELY 
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FIGURE  1.11:  EFFECT  OF  BACKBONE  TRUNK  MILEAGE  COST  VARIATIONS  ON 

OPTIMAL  NUMBER  OF  SWITCHES  FOR  AUTODIN  II  AND  RGC 
SYSTEMS  RESPECTIVELY 
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NN:  OPTIMAL  NUMBER  OF  SWITCHES 


AUTODIN  II 


AUTODIN  II:  5,300p  [$/mo  x switch] 


SWITCH  COST 


RGC : 21 , 200p  [$/mo  x switch] 


SWITCH  COST  SCALING  FACTOR 


FIGURE  1.12:  EFFECT  OF  SWITCH  COST  VARIATIONS  ON  OPTIMAL  NUMBER 


OF  SWITCHES  FOR  AUTODIN  II  AND  RGC  SYSTEMS  RESPECTIVELY 
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or  the  backbone  switch  basic  cost  have  a much  more  pro- 
nounced effect  on  the  shift  of  optimum  number  of  back- 
bone nodes  than  similar  changes  in  the  unit  backbone 
mileage  cost. 

We  would  expect  that  the  effect  of  the  unit  com- 
ponent cost  change  on  the  optimum  number  of  backbone 
nodes  has  something  to  do  with  the  magnitude  of  the 
component  cost  under  consideration.  However,  from 
Figures  1.10,  1.11,  and  1.12,  we  see  that  the  backbone 
switch  basic  cost  is  smaller  than  the  backbone  line 
mileage  cost,  yet  the  unit  switch  basic  cost  variation 
definitely  has  a stronger  effect  on  the  optimum  number 
of  backbone  nodes.  In  fact,  even  though  the  local  ac- 
cess line  mileage  cost  is  much  larger  than  the  switch 
basic  cost,  their  unit  cost  changes  have  comparable 
effect  on  the  optimum  number  of  backbone  nodes. 

After  careful  analysis  of  this  behavior  we  de- 
termined that  the  effect  of  a unit  component  cost 
variation  on  the  optimum  number  of  backbone  nodes  is 
roughly  proportional  to  the  rate  of  change  of  the  cost 
for  that  component  with  respect  to  the  number  of  back- 
bone nodes  at  the  optimum  point. 

F.  Next,  we  considered  the  effect  of  unit  component 
cost  variations  on  the  total  cost  of  the  optimum  de- 
signs produced.  We  noticed  that  the  unit  local  access 
mileage  cost  variation  produces  the  most  drastic  change 
in  the  optimum  design  cost,  the  unit  backbone  line 
mileage  cost  variation  produces  less  impact,  and  the 
unit  switch  basic  cost  variation  produces  the  least 
amount  of  change.  This  can  be  intuitively  explained 
by  the  fact  that  for  the  optimum  design,  among  these 
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three  cost  components,  the  local  access  mileage  cost  is 
always  the  largest,  the  backbone  line  mileage  cost  is 
the  second,  and  the  switch  basic  cost  is  the  smallest. 

We  can  thus  conclude  that  the  effect  of  a unit  component 
cost  variation  on  the  optimum  design  cost  is  approximate!; 
proportional  to  the  cost  of  that  component  for  the  opti- 
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1.9  COST  TRENDS  OF  SATELLITE  AND  TERRESTRIAL  NETWORK 
IMPLEMENTATIONS 


1.9.1  System  Model 

The  model  under  study  corresponds  to  the  AUTODIN  II  require- 
ments discussed  in  Section  1.8.1.  Cost  estimates  for  satellite 
space  segment  and  earth  station  are  derived,  and  various  network 
schemes  ranging  from  full  satellite  implementation  with  earth  sta- 
tions at  each  site  to  hybrid  terrestrial-satellite  configurations 
are  cost-evaluated. 

1.9.2  Trends 

The  following  main  trends  were  observed: 

1.  When  earth  stations  are  installed  at  each  site, 
the  cost  increases  very  rapidly  with  the  number  of 
user  sites  that  must  be  served.  The  cost  of  a ter- 
restrial implementation,  on  the  other  hand,  varies 
less  critically  with  number  of  sites.  Therefore, 
for  a given  value  of  throughput,  there  is  a "thres- 
hold" value  of  number  of  user  sites  above  .which  the 
terrestrial  network  is  more  cost-effective,  and  be- 
low which  the  satellite  network  is  more  cost-effec- 
tive. This  behavior  is  illustrated  by  Figure  1.13 
comparing  satellite  and  terrestrial  cost  for  a 
throughput  level  of  1.26  Mbps  (i.e.,  AUTODIN  II 
requirement) . In  particular,  for  the  AUTODIN  II 
system  with  300  distinct  locations,  the  satellite 
implementation  cost  is  $l,386K/mo,  while  the  ter- 
restrial cost  is  $898K/mo.  The  satellite  network 
with  one  earth  station  at  each  site  is  therefore 
not  cost-effective  for  AUTODIN  II. 
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FIGURE  1.13;  COMMUNICATIONS  COST  VERSUS  NUMBER  OF  SITES 
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2.  While  an  implementation  with  earth  stations  at  each 
site  may  not  be  cost-effective,  substantial  savings  may 
be  obtained  by  installing  a few  stations  located  at  ap- 
propriate centers  of  mass  (concentration  points)  of  the 
large  user  population.  In  a multilevel  network  structure, 
for  example,  considerable  savings  can  be  obtained  by  in- 
stalling earth  stations  at  each  backbone  switch,  and  re- 
placing the  backbone  terrestrial  network  with  a satellite 
network.  End-to-end  delay  performance,  however,  may  be- 
come unacceptable.  As  a solution,  hybrid  configurations 
may  be  considered  consisting  of  a "thin"  terrestrial  back- 
bone net  (for  low  delay  traffic)  and  a satellite  backbone 
net  superimposed  to  the  terrestrial  component  (for  bulk 
traffic) . A preliminary  evaluation  of  hybrid  structures 
for  the  AUTODIN  II  backbone  net  shows  that  the  cost  of 
the  hybrid  implementation  depends  on  the  number  and  lo- 
cation of  ground  stations  installed.  In  particular, 
the  minimum  cost  configuration  is  obtained  by  installing 
ground  stations  at  6 out  of  the  8 switches  of  the  pro- 
posed AUTODIN  II.  The  results  are  summarized  in  Figure 
1.14. 
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2.  MULTILEVEL  NETWORK  DESIGN;  ALGORITHMS  FOR  BACKBONE 

NODE  SELECTION 

2.1  INTRODUCTION 

In  this  study  we  are  concerned  with  the  topological  design  of 
large,  distributed  data  networks  in  general,  and  with  the  location 
of  the  backbone  switches  in  particular.  We  refer  to  a large  net- 
work as  a network  with  on  the  order  of  one  hundred  or  more  distinct 
user  installations  (each  with  one  or  more  terminals  and/or  Hosts) 
distributed  nationwide  and  with  traffic  volume  and  characteristics 
that  require  the  use  of  high  speed  backbone  trunks  (say  50  Kbps  or 
more) . 

We  distinguish  between  centralized  networks  in  which  all  traf- 
fic requirements  are  to  and  from  a central  node,  and  distributed  net- 
works in  which  traffic  requirements  are  widely  distributed  among  dif- 
ferent node  pairs  (terminal-computer  and  computer-computer) . The  em- 
phasis of  this  study  is  on  distributed  networks,  although  most  of  the 
concepts  apply  to  centralized  networks  as  a special  case. 

Finally,  we  often  distinguish  between  computer-oriented  data  net- 
works in  which  inter-computer  communications  are  predominant,  and  ter- 
minal-oriented data  networks  with  mostly  terminal-to-computer  communi- 
cations. The  concepts  presented  here  are  valid  for  both  types  of  net- 
works and  more  generally  for  networks  which  combine  the  two  types  of 

requirements. 

Large  distributed  networks  generally  result  from  the  integration 
of  existing  centralized  and/or  distributed  data  communications  systems. 
The  purpose  of  integration  is  to  achieve  the  following  advantages: 

. Line  economies. 

. Inter-system  communications  capability. 
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. Higher  network  bandwidth. 

. Improved  reliability. 

. Improved  growth  flexibility. 

. More  efficient  network  control  and  management. 

Consolidation  and  integration  of  communication  requirements  is  pre- 
sently being  considered  by  several  large  corporations  as  well  as 
overnment  and  military  agencies.  For  example,  General  Motors  is 
planning  to  combine  eight  divisional  data  networks  into  a nationwide 
corporate  network.  The  Department  of  Defense  is  in  the  process  of 
integrating  several  of  its  ADP  systems  into  a large,  hierarchical 
network . 

Specific  network  implementation  solutions  vary  considerably  de- 
pending on  the  application.  However,  a common  property  of  large,  dis- 
tributed networks  is  the  multilevel  hierarchical  structure  with  a 
backbone  network  at  the  higher  level,  and  locai  access  networks  at  the 
lower  levels.  Backbone  and  local  access  nets  may  be  further  subdivid- 
ed into  hierarchical  sublevels.  The  backbone  network  is  characterized 
by  distributed  traffic  requirements  and  is  generally  implemented  using 
the  packet-switching  technology.  Local  access  networks,  on  the  other 
hand,  have  a centralized  traffic  pattern  (all  the  traffic  is  to  and 
from  the  gateway  backbone  node)  and  are,  therefore,  implemented  with 
centralized  techniques  such  as  multiplexing,  concentration,  and  polling 

Typical  examples  of  a multilevel  structure  are  shown  in  Figures 
2.1  through  2.4.  In  Figure  2.1  each  user  device  is  point-to-point 
connected  to  the  backbone  switch.  Colocated  devices  may  be  clustered 
around  a local  TDMX,  terminal  controller  or  concentrator  and  their  traf 
fic  requirements  merged  on  a single  line  to  the  switch.  Figure  2.2 
shows  another  possible  two-level  configuration  in  which  local  access  to 
the  backbone  is  accomplished  via  a distributed  network.  This  may  be 
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the  case  of  an  international  network  interconnecting  several  national 
data  networks  [DAVIES,  1975].  Figure  2.3  shows  a local  access 
strategy  using  loop  connections.  Finally,  Figures  2.4  and  2.5  show 
examples  of  three-level  configurations.  The  structure  in  Figure  2.5 
can  also  be  viewed  as  a two- level  backbone  implementation  with  one 
level  of  local  access. 

The  most  general  multilevel  data  network  is  implemented  by 
using  at  each  level  one  (or  a combination  of)  the  following  alter- 
natives: 

. Dialup  line. 

. Point-to-point  dedicated  line. 

. Concentration. 


Time  of  frequency  division  multiplexing. 


Polling  over  a multidrop  circuit. 


Loop  or  ring. 


Packet  satellite  (multiple  access) . 


Packet  radio  (multiple  access) . 


The  selection  of  the  most  effective  network  architecture  (i.e., 
number  of  levels  and  type  of  access  at  each  level)  is  the  first 
problem  that  must  be  attacked  in  the  design  of  a large  network.  In 
this  study  we  assume  that  the  network  architecture  has  been  chosen, 
and  refer  to  Chapter  1 of  this  volume  for  a more  extensive  discussion 
of  the  available  alternatives. 
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Having  selected  the  architecture,  we  must  then  design  a cost- 
effective  topology  that  minimizes  line  and  nodal  processor  costs 
within  such  architecture.  For  the  multilevel  case  the  problem  may 
be  divided  into  two  subproblems: 

1.  Selection  of  nodal  processor  locations  for  each 

level;  and 

2.  Line  layout. 

The  two  subproblems  are  interrelated  and  should  be  solved  si- 
multaneously. However,  due  to  the  complexity  of  the  global  problem, 
an  iterative  approach  is  generally  used,  where  the  subproblems  are 
solved  separately  at  each  iteration. 

In  this  study  we  focus  on  the  backbone  switch  location  problem. 

For  a multilevel  network,  we  are  interested  in  determining  the  switch 
locations  which  minimize  the  sum  of  backbone  trunk  costs,  local  ac- 
cess costs  and  switch  costs,  yet  satisfying  well-defined  performance 
* 

requirements  (delay,  throughput,  reliability,  etc.).  For  this  purpose 
two  switch  location  algorithms  were  developed  and  tested  on  problems 
of  various  sizes.  In  particular,  the  algorithms  were  applied  to  a 
Defense  Communications  application  (AUTODIN  II)  and  to  a large  network 
example. 

In  this  report,  after  reviewing  in  Section  2.2  a series  of 
related  facility  location  problems  and  solution  techniques,  we 
discuss  the  assumptions  and  approximations  necessary  to  make  the 
problem  tractible  and  present  the  basic  problem  formulation  in 
Section  2.3.  We  then  describe  three  alternative  solution  methods 
and  discuss  their  computational  complexity  in  Sections  2.4,  2.5  and 
2.6.  Two  large  network  applications  are  presented  in  Section  2.7  and 
general  properties  of  the  optimal  node  location  strategy  are  discussed. 
In  Section  2.8,  the  solution  approach  is  extended  to  more  general 
problems . 


2.9 


Network  Analysis  Corporation 


2.2  RELATED  PROBLEMS  AND  SOLUTION  TECHNIQUES 

There  are  many  problems  that  are  related  to  the  backbone  node 
location  problem,  including  warehouse  location  problems,  clustering 
problems,  partitioning  problems  and,  in  the  data  communications 
area,  concentrator  location  problems.  In  this  section,  we  discuss 
these  problems,  their  solution  techniques,  and  the  applicability  of 
these  techniques  to  the  backbone  node  location  problem. 

2.2.1  Warehouse  Location  Problems 


The  warehouse  location  problem  may  be  briefly  defined  as  the 
determination  of  the  number,  location,  and  capacity  of  source  sites 
in  order  to  minimize  the  cost  of  satisfying  a set  of  shipping  re- 
quirements under  a given  cost  matrix  [COOPER,  1962] . Efroymson  and 
Ray  present  a Branch  and  Bound  Algorithm  solution  technique  [EFROYMSON, 
1966],  which  can  be  applied  to  the  general  backbone  node  location 
problem.  However,  this  approach  has  the  drawback  of  being  com- 
putationally prohibitive  for  large  problems. 

The  computational  requirements  for  obtaining  exact  solutions  to 
these  problems  have  given  rise  to  many  heuristic  approaches.  Among 
the  more  successful  is  the  "Add"  Algorithm  [KUEHN,  1963].  In  this 
approach,  a star  network  with  all  "customers"  directly  connected  to 
a central  warehouse  is  assumed  to  start  with.  Each  possible  ware- 
house location  is  then  evaluated  by  determining  the  cost  reduction 
which  would  be  achieved  by  placing  a warehouse  at  the  location.  The 
location  giving  the  greatest  reduction  is  then  selected  for  the 
first  placement  of  a warehouse.  With  the  new  warehouse  in  place, 
the  process  is  repeated  for  the  next  location.  When  no  further  cost 
reduction  is  achieved,  the  process  halts.  This  approach  however  is 
not  directly  applicable  to  the  backbone  node  location  problem,  since 
in  the  latter  there  is  no  clear  notion  of  central  warehouse  for  line 
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saving  evaluation.  An  additional  drawback  of  the  Add  is  that  each 
time  a site  is  selected,  it  becomes  permanent,  but  there  is  little 
reason  to  think  that  the  location  of  the  best  single  backbone  node 
is  also  one  of  the  best  locations  for  two  backbone  nodes,  etc. 

The  "Drop"  Algcrithm  [FELDMAN,  1966]  is  basically  the  reverse 
of  the  "Add"  Algorithm.  Warehouses  are  initially  assumed  to  be  at 
all  possible  locations,  and  , "customers"  are  assigned  to  multiple 
warehouses.  The  procedure  is  to  then  eliminate  the  warehouse  whose 
elimination  most  reduces  the  cost.  The  process  is  repeated  until  no 
further  reduction  is  possible.  The  difficulties  here  are  directly 
analogous  to  those  of  the  "Add"  Algorithm. 

Other,  less  promising  formulations  of  the  warehouse  location 
problem  and  related  solution  techniques  are  contained  in  [TEITZ, 
1967],  [GOLDMAN,  1970],  [ELZINGA,  1971],  [GOLDMAN,  1971]. 
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Clustering  Problems 


Basic  aspects  of  the  backbone  node  location  problem  can  be  posed 
in  a clustering  context.  The  clustering  problem  may  be  thought 
of  as  detecting  inherent  separations  between  subsets  of  a given 
point  set,  where  the  separations  may  be  in  terms  of  several  dif- 
ferent measures.  In  the  backbone  node  location  problem.,  one  might 
expect  some  measure  to  be  available  such  that  nodes  clustered  under 
this  measure  would  most  appropriately  be  connected  to  the  same  node. 

Many  clustering  techniques  exist  [AUGUSTON,  1970],  [BONKER,  1964], 

[GOWER,  1969],  [LING,  1973],  [PATRICK,  1969];  however,  few  appear  to 
suggest  measures  which  may  be  applicable  to  the  node  location  problem. 
Among  the  more  promising  are  those  which  attempt  to  cluster  points  on 
a plane  [ZAHN,  1971],  [JARVIS,  1973].  Zahn  attempts  to  identify  ges- 
talt clusters  (two  dimensional  point  sets  naturally  perc »ived  as 
separate  groupings)  by  connecting  the  points  with  a minimum  spanning  tree, 
and  deleting  relatively  long  brances  to  form  components,  or  clusters. 
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Jarvis  and  Patrick  offer  a similarity  measure  based  on  shared  near 
neighbors  to  generate  "nonlobular"  clusters.  This  technique  and 
that  of  Zahn  have  much  in  common.  However,  neither  of  the  approaches 
appear  to  offer  more  than  insight  to  the  complexity  of  the  backbone 
node  location  problem. 
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2.2.3  Partitioning  Problems 

The  node  location  problem  may  be  viewed  as  that  of  partitioning 
a set  of  elements  in  a way  that  optimizes  some  objective  function 
defined  on  the  set  of  all  partitions.  The  partitioning  may  be  in 
terms  of  nodes  connected  to  the  same  access  facility,  and  the  objective 
function  is  simply  network  cost.  There  are  several  formulations  to 
the  partitioning  problem,  in  terms  of  set  theory  [GARFINKEL,  1968], 
[ROACH,  1972],  and  graph  theory  [JENSEN,  1971],  [KERNINGHAN,  1970], 
[KERNINGHAN,  1971].  Perhaps  the  most  interesting  of  these  formu- 
lations, from  the  node  location  perspective,  is  the  formulation  by 

Roach  in  which  the  objective  function  is  to  minimize  the  maximum 
within-cluster  distance.  Although  the  solution  technique  offered  is 
one  of  forming  reduced  subproblems,  and  is  somewhat  improved  over 
integer  programming,  it  is  still  far  too  computationally  complex  to 
be  of  interest  in  our  problem. 
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Concentrator  Location  Problems 
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The  concentrator  location  problem  is  very  similar  in  nature 
to  the  backbone  switch  problem.  A typical  design  strategy  for 
large  teleprocessing  networks  consists  of  connecting  terminals  to 
concentrators  via  multidrop  circuits,  and  of  connecting  concentra- 
tors to  the  Host  vi*  dedicated  lines.  There  is  a constraint  on  the 
maximum  number  of  terminals  and  traffic  volume  allowed  per  multidrop 
circuit.  Also,  there  is  a constraint  on  the  maximum  number  of 
circuits  that  can  be  derived  from  a concentrator.  The  concentrator 
location  problem  consists  of  selecting  the  number  and  location  of 
concentrators  which  minimize  the  total  cost  (line  and  processor)  yet 
satisfying  the  constraints. 
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The  problem  has  received  considerable  attention  in  the  data 
communication  network  literature  [BAriL,  1972],  [CAHIT,  1973],  [CHOU, 
1973],  [DOLL,  1971],  [FRANK,  1972],  [GREENBERG,  1973],  [TANG,  1973], 
[WOO,  1973].  The  approaches  have  almost  all  been  heuristic  in 
nature,  and  most  have  dealt  with  only  the  less  general  version  of 
the  problem  where  only  point-to-point  connections  to  a concentrator 
are  permitted.  For  this  simpler  problem,  two  attractive  approaches 
seem  to  be  a direct  adaptation  of  the  "Add"  Algorithm  of  the  ware- 
house location  problem,  and  a "graceful"  Drop  Algorithm  based  on 
link  removal  rather  than  concentrator  removal  [BAHL,  1972].  The 
former  seems  to  be  computationally  considerably  more  efficient,  but 
the  latter  seems  to  yield,  in  general,  slightly  better  results.  An 
approach  based  on  combinatorial  optimization  over  certain  selected 
subsets  of  the  network  has  been  offered  by  Greenberg  [GREENBERG, 
1973],  and  has  the  attraction  of  partitioning  the  main  problem  into 
subproblems  in  an  iterative  manner,  and  reoptimizing  the  partitions 
based  on  global  considerations.  However,  the  basic  heuristic  used 
in  the  partition  process  does  not  restrict  the  size  of  partition 
elements;'  the  procedure  for  optimization  is  based  on  enumeration 
(and  is  thus  very  costly) , and  the  approach  is  again  for  point-to- 
point  problems. 

The  general  concentrator  location  problem  has  been  attacked  by 
Woo  and  Tang  [WOO,  1973]  with  an  algorithm  that  approximates  the 
problem  with  a simpler  point-to-point  problem,  in  which  the  point- 
to-point  cost  of  connecting  two  nodes  is  a weighted  average  of  the 
direct  connection  cost  and  the  shared  cost  of  connecting  through  a 
minimum  spanning  tree.  We  call  this  the  Average  Tree-Direct  (ATD) 
lgorithm.  After  the  problem  simplification,  the  "Add"  Algorithm  is 
applied  to  make  site  selections.  Recall  that  in  the  "Add"  Algo- 
rithm, all  candidate  sites  are  evaluated  for  savings,  the  best 
selected,  and  with  assigned  nodes,  removed  from  further  considera- 
tion. The  procedure  is  iterated  until  no  new  site  is  found  to  give 
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positive  savings.  A vigorous  proof  of  the  following  theorem  which 
justifies  the  termination  condition  for  the  "Add"  Algorithm  has  been 
reported  [WOO,  1973] s 

Theorem: 

Let  L,  L^,  and  L^2  be  optimal  network  assignments  corresponding 
respectively  to  concentrator  sets  (jQ,  ji+1), 

and  ji+1,  ji+2)»  and  let  C(L*'  C(L1),  and  C(L12)  be 

their  respective  costs.  Then  C(L)  - CfL^  _>  CtL^)  - C(L12). 

Note  that  this  theorem  does  not  say  that  the  network  cost  is  a 
convex  function  of  the  number  of  concentrators,  but  rather  that  the 
cost  is  convex  over  the  iterations  of  adding  a new  concentrator  to 
an  existing  set  of  concentrators  without  reoptimization  of  the  ex- 
isting locations.  In  fact,  counter  examples  are  easily  produced 
in  which  the  cost  is  not  convex  over  the  number  of  concentrators 
when  the  given  number  of  concentrators  are  optimally  located  in 
each  case. 

An  algorithm  with  near  optimality  comparable  to  the  ATD  Algor- 
ithm, but  computationally  much  more  efficient  is  the  COM  (Center  of 
Mass)  Algorithm  proposed  by  McGregor  and  Shen  [McGREGOR,  1975] . In 
a preliminary  pass,  the  algorithm  creates  clusters  with  a constraint 
on  the  number  of  terminals  or  total  traffic  per  cluster.  Each 
cluster  is  then  replaced  by  its  center  of  mass  (calculated  on  the 
basis  of  appropriate  terminal  weights) , and  concentrator  locations 
are  selected  with  the  Add  Algorithm,  so  as  to  minimize  star  con- 
nection cost  from  center  of  mass  to  concentrator.  The  basic  notions 
used  in  developing  the  COM  Algorithm  are  the  fact  that  clusters  can 
be  defined  independently  of  the  actual  concentrator  locations  chosen 
and  that  the  cost  of  connecting  the  nodes  in  a cluster  to  a con- 
centrator via  a multidrop  circuit  can  be  reasonably  approximated 
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with  a constant  (which  depends  solely  on  the  cluster)  plus  the  cost 
of  the  point-to-point  connection  from  center  of  mass  to  concen- 
trator. 

The  concentrator  location  problem  is  quite  resemblant  to  the 
backbone  location  problem.  As  a main  difference , however,  concen- 
trators are  point-to-point  connected  to  the  central  computer,  while 
backbone  nodes  are  interconnected  in  a distributed  network.  Con- 
sequently, the  cost  of  connecting  a candidate  to  the  backbone  net 
cannot  be  determined  in  principle  until  all  the  switches  have  been 
selected  and  the  backbone  topology  designed.  To  overcome  this 
difficulty,  appropriate  backbone  line  cost  approximations  are  intro- 
duced and  are  discussed  in  Section  2.3. 
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2.3  BASIC  APPROACH  AND  PROBLEM  FORMULATION 


A very  general  formulation  of  the  large  network  design  problem 
is  the  following: 


. Hierarchical  structure  (i.e.,  number  of  hierarchical 
levels  and  structure  of  each  level) . 


Terminal  and  Host  locations 


Point-to-point  traffic  requirement  matrix 


. Delay  requirements  (possibly  different  requirements 
for  different  users) . 


. Cost  and  characteristics  of  the  communications 
hardware  (e.g.,  packet  switching  processor,  concentrator, 
TDMX,  etc.)  for  each  hierarchical  level. 


. Candidate  locations  for  communications  hardware 
installation  (generally,  different  sets  of  candidate 
locations  for  different  hardware  types  and/or  hierar 
chical  levels) . 


Line  tariff  alternatives 
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Minimize:  Total  communications  cost  D: 

D = line  cost  + communications  hardware  cost. 

Over  The  Variables: 

. Communications  hardware  locations. 

. Line  layout. 

. Line  speed  assignment. 

Such  that: 

. Traffic  requirements  are  met. 

. Delay  constraints  are  met. 

. Communications  hardware  capacity  is  not  exceeded. 

. Reliability  constraints  are  met. 

2.3.2  Basic  Approach 

The  general  problem  is  rather  complex  to  solve,  especially  if 
the  local  access  subnetworks  have  a multilevel  structure,  or  re- 
quire some  kind  of  topological  optimizations  (e.g.,  loop,  multidrop, 
distributed  packet  switch  configurations,  etc.).  To  obtain  subop- 
timal  solutions  we  propose  an  iterative  approach  in  which  the  global 
problem  is  decomposed  into  subproblems  of  smaller  site,  and  each 
subproblem  is  solved  separately. 
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The  iterative  procedure  is  the  following: 

0.  Let: 

n = o 

D(o)  = °°  (where  D(i)  is  the  total  cost 
at  iteration  i) 

1.  Perform  preliminary  clustering  of  the  user  in- 
stallation (Hosts  and  terminals),  such  that  the  re- 
quirements in  each  cluster  (i.e.,  traffic  volume,  num- 
ber of  terminals  and/or  Hosts,  etc.)  do  not  exceed  the 
capacity  of  the  intermediate  concentration  devices 
planned  for  the  second  level  of  the  hierarchy  (e.g., 
concentrators,  TDMX , front  end  processors,  etc).  For 
each  cluster  identify  the  center  of  mass  (COM)  using 
as  weights  for  terminals  and/or  Hosts  the  line  speed 
requirements.  Locate  second  level  concentrators  at 
the  COM  sites. 


2.  Let  n = n+1. 

3.  Determine  the  lowest  cost  backbone  switch  location 
strategy  for  the  two-level  network  consisting  of:  (a)  con- 
nections from  second  level  concentrators  to  backbone 
switches;  (b)  backbone  switches;  and  (c)  backbone  trunks. 
Number  and  location  of  backbone  switches  are  selected 

so  as  to  minimize  the  sum  of  the  costs  of  components  (a) , 
(b) , and  (c) . 

4 . Based  on  the  backbone  switch  location  strategy  in 
(3) , optimize  the  topology  of  the  local  access  (multi- 
level) subnetworks.  Let  D(n)  be  the  total  network  cost 
(backbone  + local  access) . 
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5.  If  (D(n-l)  - D(n)}  < e stop:  a local  minimum 

was  obtained.  Otherwise  go  to  (3) . 

Since  D(n)  is  monotonically  decreasing,  the  procedure  converges 
to  a locally  optimal  topology  with  cost  D £ 0.  The  final  topology 
is  locally  optimal  with  respect  to  perturbations  of  number  and  lo- 
cation of  switches  and/or  lower  level  concentration  devices. 

Several  different  local  minima  may  be  generated  by  changing 
the  operational  parameters  of  the  clustering  routine  in  Step  (1) 
and  the  backbone  switch  routine  in  Step  (3) . The  lowest  cost  so- 
lution among  all  the  local  minima  generated  may  be  regarded  as  the 
suboptimal  solution  to  our  problem. 

The  critical  (computationwise)  routines  in  the  above  procedure 
are  the  following: 

a.  Identification  of  the  COM's  in  Step  (1); 

b.  Backbone  switch  location  in  Step  (3) ; 

c.  Backbone  topology  design  in  Step  (3);  and 

d.  Local  access  design  in  Step  (4). 

The  partitioning  of  a large  terminal  population  into  "minimal 
cost"  clusters  satisfying  given  constraints  is  a classical  problem 
in  data  network  design.  An  efficient  technique  for  clustering  and 
COM  identification  in  a population  of  multidropped  terminals  was 
presented  in  (McGREGOR,  1975].  Similar  techniques  may  be  constructed 
for  other  local  access  strategies. 

Several  algorithms  for  the  design  of  local  access  topologies 
are  found  in  the  literature.  Typically,  the  algorithms  solve  the 
problem  of  optimal  location  of  concentrators,  and  optimal  layout 


t 
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of  multidrop  lines  to  connect  terminals  to  concentrators.  A review 
of  the  most  popular  algorithms  was  presented  in  Section  2.2.  Such 


algorithms  can  be  extended  to  the  design  of  more  general  multilevel 
local  access  structures. 

For  the  design  of  the  backbone  network,  efficient  heuristics 
were  reported  in  [FRANK,  1970],  [GERLA,  1974],  [GERLA,  1975]  and 
[LAVIA,  1975] . A useful  approximation  for  estimating  total  backbone 
cost  is  discussed  in  the  sequel. 

As  opposed  to  the  previous  problems  that  have  been  well  dis- 
cussed in  the  literature,  the  problem  of  selecting  number  and  lo- 
cation of  backbone  switches  in  a distributed  network  is  a challeng- 
ing problem  that  has  received  little  attention  in  the  past.  In  this 
study  we  intend  to  present  techniques  for  its  solution  and  to  inves- 
tigate the  property  of  such  solutions.  To  do  this,  we  focus  on  Step 
(3)  of  the  global  design  procedure  and  consider  the  design  of  net- 
works with  two  hierarchical  levels  and  point  to  point  connections 
from  user  sites  to  backbone  switches  (see  Figure  2.1).  The  results 


are  then  generalized  to  multilevel  structures. 
2.3.3  Backbone  Switch  Location  Problem 


j 
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. Line  tariff  alternatives. 

Minimize: 

Total  communications  cost  D; 

where  D = trunk  cost  + switch  cost  + local  access 
to  line  cost. 

Over  The  Variables: 

. Switch  number  and  location. 

. Backbone  topology. 

Such  that: 

. Backbone  traffic  requirements  are  met. 

. Backbone  delay  constraints  are  met. 

. Backbone  reliability  constraints  are  met. 


. Switch  capacity  constraints  are  met. 

2.3.4  | Simplifying  Assumptions 

The  following  assumptions  and  approximations  simplify 
sign  procedure: 

. Switch  cost  is  assumed  linear  with  capacity. 

. Backbone  trunk  cost  is  calculated  using  a direct 

distance  model. 
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2. 3. 4.1  Switch  Cost  Model 

In  a large  network  implementation  switches  are  often  modular 
in  order  to  allow  for  expandibility  and  growth  flexibility.  The 
switch  capacity  can  therefore  be  increased  in  fixed  size  steps,  and 
the  cost  is  a stepwise  function  of  capacity.  To  simplify  the  pro- 
blem we  assume  that  switch  capacity  is  a continuous  variable,  and 
switch  cost  is  the  sum  of  two  terms:  a fixed  cost,  which  accounts 
for  building  costs,  operation  and  maintenance  costs  etc. r and  a vari- 
able cost,  which  varies  linearly  with  capacity.  This  approximation 
is  very  good  in  the  case  of  highly  modularized  switches  such  as  the 
Pluribus,  for  example.  For  less  modular  switch  architectures  (e.g., 
a cluster  of  identical  processors)  the  cost  tends  to  increase  more 
rapidly  than  capacity. 

The  continuity  and  linearity  of  switch  cost  as  a function  of 
capacity  implies  that,  for  a given  selection  of  backbone  switches, 
the  sum  of  local  access  and  backbone  switch  cost  is  minimized  by  as- 
signing each  terminal  site  to  the  nearest  backbone  switch  (nearest 
neighbor  assignment) . 

For  non-linear  switch  cost  models  the  nearest  neighbor  assign- 
ment is  in  general  non-optimal,  and  more  elaborate  algorithms  for  ter- 
minal to  switch  assignment  are  required.  One  of  the  backbone  selection 
methods  discussed  in  the  sequel,  namely  the  ADD  method,  provides  such 
an  assignment  as  a byproduct  for  the  stepwise  cost-capacity  case. 

2 . 3 . 4 . 2 Backbone  Network  Design 

Since  the  optimal  node  location  strategy  is  the  result  of  the 

tradeoff  between  local  access  cost  and  backbone  cost,  the  backbone 
topology  should  in  principle  be  redesigned  at  each  iteration  to  eval- 
uate the  backbone  cost.  This  approach  however  is  computationally  too 
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time  consuming  for  those  algorithms  which  at  each  iteration  require 
the  cost  evaluation  and  comparison  of  several  backbone  configura- 
tions corresponding  to  different  candidate  node  selections.  To  this 
end  we  recall  that  distributed  design  algorithms  are  typically 
iterative  and  offer  a tradeoff  between  accuracy  and  number  of  itera- 
tions performed.  If  only  a limited  number  of  iterations  are  used  to 
reduce  the  computational  burden/  the  solution  may  not  be  very  accurate. 
In  the  limit,  the  size  and  irregularity  of  the  error  may  invalidate 
the  line  savings  comparison  between  alternative  location  strategies. 

We  need  therefore  an  approximate  design  technique  with  the  fol- 
lowing properties: 

1.  Computational  efficiency;  and 

2.  Consistency  (in  the  sense  that  the  error  intro- 
duced is  "systematic",  without  severe  jumps  corres- 
ponding to  perturbations  in  switch  number  and  location) . 

The  technique  need  not  be  very  accurate  in  the  absolute  sense, 
since  only  cost  variations  relative  to  insertion  or  removal  of  one 
node  at  a time  are  considered  at  each  step  of  the  backbone  location 
algorithm. 

A rather  naive  backbone  line  cost  evaluation  technique,  which 
however  satisfies  our  requirements,  is  the  Direct  Distance  Approxima- 
tion (DDA)  described  be^ow.  First,  we  make  the  following  assumptions: 

1.  Monthly  trunk  cost  is  linear  with  bandwidth  and 
mileage  (i.e.,  c = cost/mile  x unit  bandwidth  x month  = 
constant) . 

2.  Trunks  are  fully  utilized. 

3.  The  network  topology  is  fully  connected. 
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Using  the  above  assumptions  the  total  backbone  trunk  cost  D can  be 
expressed  as  follows: 

NN  NN 

D = Z Z c r. . d. . (1  + B)  (2.1) 

i=l  j=l  13  13 

where:  r^j  = Traffic  requirement  from  i to  j 

d^j  = Direct  distance  from  i to  j 
b = Line  overhead  (protocols,  etc.) 


NN  = Number  of  switches 


The  cost  approximation  in  Equation  (2.1)  provides  a simple  and  con- 
sistent relationship  between  cost,  geographical  location  and  traffic 
requirements  of  the  various  candidates. 

A more  realistic  approximation  can  be  obtained  by  including  in 
the  model  the  following  factors: 

1.  Non-direct  routing  penalty  P defined  as  the  ratio 

between  shortest  path  distance  and  direct  distance,  for 
the  average  source  - destination  pair  (P  > 1). 

2.  Average  link  utilization  9 (where  9 < 1) . 

3.  Fixed  cost  per  node  F = (average  number  of  line 
terminations  per  node)  x (line  termination  cost.) 


By  introducing  the  above  coefficients  in  Equation  (2.1)  we 
obtain: 


Z 

j=l 


c ri1  d±i  (1  + b)P 
9 


D 


+ NNxF 


(2.2) 
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The  coefficients  P,  ? and  P are  determined  experimentally  for 
various  values  of  NN,  under  the  assumption  that  such  coefficients 
depend  solely  on  NN,  and  are  insensitive  to  perturbation  of  switch 
locations.  For  the  evaluation  of  the  coefficients  we  design  a low 
cost  topology  with  NN  arbitrarily  chosen  switches  and  calculate  the 
corresponding  communications  cost  D.  Letting  NA  be  the  number  of 
lines,  we  have  the  following  expression  for  the  coefficients: 

„ 2 x NA  x T 

F = NN 

D - F x NN 


The  cost  estimate  for  a new  network  design  requires  on  the 
2 

order  of  NN  operations,  while  it  takes  only  NN  operations  to  update 
the  cost  estimate  after  the  addition  or  deletion  of  a single  node. 
This  is  a considerable  computational  saving  with  respect  to  con- 
ventional design  algorithms  that  require  on  the  order  of  NN3 

operations  for  each  iteration. 

* 

The  DDA  technique  defined  above  assumes  a linear  relationship 
between  line  cost  and  bandwidth.  In  some  cases  this  assumption  is 
well  verified  (e.g.,  backbone  channels  implemented  with  several 
lines  of  same  speed  in  parallel) . If  multiple  line  speed  options 
are  used,  the  cost-bandwidth  function  may  be  nonlinear,  in  which 
case  some  non  trivial  modifications  to  the  basic  DDA  may  be  nec- 
essary. 

Numerical  results  are  presented  in  the  application  section  to 
show  the  degree  of  accuracy  that  can  be  obtained  with  the  DDA 
technique. 
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2.4  CLUSTER  ALGORITHM 
2.4.1  3asic  Objectives 

It  is  the  general  characteristic  of  Heuristic  Network  Design 
Algorithms  that  they  can  always  be  improved,  with  additional  de- 
velopment expense.  Furthermore,  with  optimal  implementations,  there 
is  a tradeoff  between  the  cost  of  the  network  designed  (i.e.,  the 
more  iterations  of  the  algorithm  with  larger  execution  time,  the 
more  refined  the  resultant  design) . The  basic  objectives  governing 
development  of  the  design  procedure  presented  in  this  section  were: 

* 

• Low  development  cost; 

• Ease  of  use; 

• Low  use  cost;  and 

• Production  of  reasonable  designs. 

Thus,  the  procedure  is  a simple,  easy  to  implement  approach  to 
a complex  problem  intended  to  enable  a user  to  easily  develop  rea- 
sonable "ball-park"  designs  with  little  expense  for  the  design 
effort. 


2.4.2  Strategy  Of  The  Design  Procedure 

To  accomplish  the  objectives  outlined  above,  the  network  de- 
sign problem  is  divided  into  three  parts: 
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Determine  N,  the  optimum  number  of  backbone  nodes 


2.  Given  N,  the  optimum  number  of  backbone  nodes,  determine 
the  optimum  N locations  at  which  to  place  the  nodes. 


3.  Given  the  optimum  number,  and  location  of  backbone 
nodes,  design  the  backbone  network. 


The  Heuristic  Algorithm  presented  in  this  section  is  designed 
to  determine  answers  for  the  second  part.  It  takes  as  input  the 
usual  network  database,  tariff  information,  etc.,  and  also  a spec- 
ification of  N as  the  number  of  backbone  locations  to  be  determined. 
The  output  is  simply  a set  of  N locations  which  are  heuristically 
determined  to  be  the  "best."  The  output  of  the  Heuristic  Algorithm  is 
then  processed  in  Part  3 to  determine  the  total  network  cost,  usu- 
ally with  a simple  design  program  that  in  fact  yields  estimates  of 
the  total  cost  rather  than  detailed  designs.  This  enables  the  net- 
work designer  to  determine  the  best  value  of  N by  iteratively  ap- 
plying the  Heuristic  Algorithm  and  cost  estimating  routine  over  a 
range  of  N.  Thus,  for  each  value  of  N examined,  a total  network 
cost  is  determined  and  the  sensitivity  of  cost  to  variations  in  N 
can  be  examined.  Having  determined  the  value  of  N that  gives  mini- 
mum cost,  a more  refined  design  program  can  be  applied  to  the  out- 
put of  the  Heuristic  Algorithm  to  procure  a detailed  design. 

The  success  of  the  above  procedure  depends  on  the  ease  in 
which  a total  network  design  cost  can  be  determined  for  a given 
value  of  N,  and  the  "goodness"  of  the  final  design  resulting  from 
application  of  the  process.  The  ease  and  effectiveness  of  applying 
the  process  as  outlined  above  must  be  evaluated  in  terms  of  the  de- 
signer's time  and  the  design's  cost  in  comparison  to  applying  a 
more  refined  procedure. 
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In  the  subsections  which  follow  an  overview  of  the  Heuristic 
Algorithm  for  developing  answers  to  Part  2 is  presented  followed 
by  a detailed  description  of  the  algorithm.  The  computational  com- 
plexity of  the  algorithm  is  then  considered.  In  Section  2.7  of  this 
chapter,  various  experimental  results  of  applying  the  algorithm  are 
reported. 

jf.  I 

2.4.3  Overview  of  Algorithm 


\ 

■ 


F 
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The  general  problem  of  interest  was  formulated  in  Section  2.3.3, 
and  in  the  previous  section  the  strategy  of  dividing  the  problem 
into  three  distinct  parts  was  presented.  The  first  part  was  de- 
signer-oriented, and  the  last  part  is  the  classical  topological 
design  problem  for  which  a simple  cost  estimation  routine  is  pre- 
sented in  Section  2. 3. 4. 2.  The  second  part,  selection  of  the  loca- 
tions for  a given  number  of  backbone  nodes,  is  the  subject  of  the 
algorithm  described  in  this  subsection.  This  reduced  part  of  the 
problem  can  be  given  its  own  problem  formulation  as  follows: 

Given: 

• Terminal  and  Host  locations. 

• Point-to-point  traffic  requirements. 

• Backbone  switch  cost  and  characteristics. 


Set  of  candidate  sites  for  switch  installation. 
N,  the  number  of  backbone  switches  to  be  used. 
Line  tariff  alternative. 
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The  location  of  the  N backbone  switches  that  enables  design  of 
a network  with  minimum  communications  cost  D,  where  D = trunk  cost  + 
switch  cost  + local  access  line  cost,  and  whew  the  network  must 
satisfy: 


Note  that  in  this  problem  formulation,  the  constraints  and 
variables  of  the  problem  formulation  in  Section  2.3.3  are  reflected 
in  the  objectives  for  the  final  network,  but  not  as  part  of  this 
problem  in  particular.  Thus,  the  problem  as  formulated  above  is 
simply  to  select  N sites;  it  is  not  to  produce  a network  design. 

The  N sites  must  be  selected  in  such  a way  as  to  ensure  their  ef- 
fectiveness as  locations  for  backbone  switches,  but  there  is  no 
explicit  requirement  to  relate  the  switch  locations  to  the  network 
topology,  cost,  or  performance.  Such  relations  are  developed  in 
Part  3 of  the  problem  where  the  network  design  is  produced. 

Tht.  approach  taken  to  this  problem  is  based  on  the  following 
rather  simple  and  optimistic  assumption: 


Assumption  2.4.1 


Backbone  switch  locations  that  minimize  local  access  cost  also 
minimize  total  network  cost. 
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The  assumption  is  not  without  rationale,  although  it  is  clearly 
a blatant  simplification  of  a complex  problem.  The  basic  strength 
and  weakness  of  the  assumption  are  illustrated  by  the  example  shown 
in  Figure  2.6a.  Two  sets  of  points  each  equally  distributed  on  a 
circle  of  radius  one  are  separated  in  distance  by  an  amount  x>>1. 

If  all  points  are  candidate  sites  for  backbone  nodes,  and  if  two 
backbone  nodes  are  to  be  used  (N=2) , then  points  d and  h are  clearly 
optimum,  giving  a total  line  length  of  D=  = x + 9.66.  These  two 

a 

points  also  clearly  minimize  the  local  access  length*  however, 
selection  of  any  one  point  in  each  set  will  minimize  the  local  ac- 
cess length.  Thus,  an  algorithm  based  on  the  assumption  would  in 
this  case  select  a point  in  each  set,  making  the  correct  global 
decision,  but  would  equally  likely  select  any  points  (such  as  b and 
f)  as  it  would  select  the  optimum  points  (d  and  h) . Thus,  the 
strength  is  the  tendency  to  be  correct  to  the  extent  that  it  at 
least  optimizes  over  one  cost  component,  and  its  weakness  is  that 
there  is  no  consideration  of  the  tradeoffs  in  the  selection  with 
the  cost  of  the  backbone  topology.  Local  access  cost  and  backbone 
cost  are-*  invariably  negatively  correlated:  minimizing  one  ensures 

that  the  other  is  not  minimized  (e.g.,  in  Figure  2.6a  selection  of 
points  c and  d as  backbone  nodes  would  minimize  the  backbone  cost) . 
However,  since  the  total  cost  incorporates  the  sum  of  the  two  ele- 
ments, if  one  is  dominant  (i.e.,  deviation  from  its  minimal  cost 
configuration  increases  cost  faster  than  cost  is  reduced  in  the 
other)  then  it  is  sufficient  to  seek  minimization  of  only  the  one 
element.  The  assumption  is  simply  that  in  most  real  cases,  the 
local  access  cost  is  indeed  dominant  over  the  backbone  cost.  Fig- 
ure 2.6b  shows  an  example  of  such  a situation.  The  point  configura- 
tion of  Figure  2.6a  is  increased  to  include  a point  at  the  center 
of  each  circle.  The  selection  of  two  backbone  nodes  at  i and  j is 
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FIGURE  2.6;  EXAMPLES  OF  LOCAL  ACCESS  COST  AND  TOTAL  COST  DEPENDENCIES 
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a unique  minimum  of  local  access  length  (db=8) . The  total  length 
is  = x + 10.  Alternatively,  selection  of  points  d and  h as  be- 
fore gives  a local  access  cost  of  d^  = 6 + 4/2  and  a total  length 
of  Db  = x + 11.65. 

The  assumption  2.4.1  greatly  simplifies  the  problem:  select 

the  N sites  which  minimize  local  access  cost.  However,  it  does 
not  change  the  basic  character  of  the  problem;  it  is  still  a "re- 
source allocation"  problem  similar  in  structure  to  the  various 
problem  types  outlined  in  Section  2.2.  The  approach  to  the  problem 
taken  here  is  based  on  the  following  elementary  relation: 


Given  a set  of  points  <p  = { (X^  Y^)  } with  each  point  having  a 
weighing  factor  m. , the  point  (X  , Y ) defined  by: 


minimizes  the  function 


This  relation  is  simply  the  center-of-mass  concept  from  elementary 
physics.  However,  if  the  weighing  factors  are  interpreted  as  the 
cost  per  unit  distance,  then  D is  simply  the  local  access  cost  for 
the  set  of  points  4>,  and  thus  the  point  (Xc,  Yc)  is  the  optimal  lo- 
cation of  the  backbone  switch  for  the  case  N = 1.  Similarly,  for 
N greater  than  1,  if  4>  is  divided  into  subsets  $ . , j=l,...,N,  then 
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the  center  of  mass  point  in  each  subset  is  the  best  backbone 
switch  location  for  that  subset.  Thus,  given  a partition  of  the 
points  $ into  N subsets,  where  each  subset  must  have  one  backbone 
switch,  then  the  center  of  mass  of  each  subset  defines  the  back- 
bone switch  location  which  minimizes  the  local  access  cost.  Con- 
sequently, the  problem  now  becomes  one  of  optimally  partitioning 
the  set  4>  into  N subsets.  The  approach  to  this  problem  is  based 
on  the  above  relation  and  the  following  assumption: 

Assumption  2.4.2 

natural  geographic  groupings  of  nodes  are  a good  basis  for 
partitioning. 

The  assumption  may  be  viewed  as  the  heuristic  basis  for  pursu- 
ing a gestalt  clustering  approach  to  the  partition  problem  as  de- 
fined above.  The  clusters  are  intended  to  reflect  natural  group- 
ings of  nodes  that  can  be  most  appropriately  served  by  single  nodes 
at  their  center  of  mass.  The  clusters  are  formed  by  "rolling 
snowballs"  in  a rather  "balanced"  fashion.  First,  the  two  nodes 
closest  together  are  selected.  These  nodes  are  then  replaced  by 
a single  node  at  their  "center  of  mass"  with  the  combined  weighing 
factors  of  the  first  two  nodes.  The  merging  process  continues  on 
a closest  node  pair  basis  until  only  N nodes  remain.  These  N 
nodes,  being  the  center  of  mass  of  their  respective  clusters,  are 
the  selected  locations  for  the  N backbone  switches. 

Unfortunately,  the  approach  outlined  above  is  a little  too 
simple  to  produce  a set  of  N good  locations  for  the  backbone 
switches.  The  primary  deficiency  is  the  possible  dramatic  dif- 
ference in  size  of  the  clusters,  which  in  practical  situations 
usually  results  in  poor  designs.  To  compensate  for  this  tend- 
ency a parameter  Z is  defined  as  a cluster  capacity,  and  a para- 
meter a is  defined  as  a size  threshold  expressed  as  a percent  of 
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Z.  Clusters  will  be  grown  until  they  reach  a size  greater  than  aZ 
and  then  they  will  be  stopped.  Clusters  are  not  permitted  to  merge 
if  the  merger  exceeds  Z.  The  first  N cluster  to  be  stopped  will  be 
selected  as  the  backbone  nodes.  Note  that  this  procedure  is  less 
computationally  costly  than  the  previous  as  it  does  not  complete  the 
partitioning  process.  The  partitioning  could  easily  be  completed  by 
now  requiring  each  remaining  cluster  to  be  merged  to  its  closest  one 
of  the  N selected  clusters.  The  resulting  N selected  backbone 
switch  locations  would  then  be  the  center-of-mass  points  of  the 
partition  elements.  However,  for  expediency,  the  procedure  adopted 
here  is  to  s imply  select  the  center  of  mass  points  of  the  first  N 
clusters  reaching  a size  greater  than  aZ.  The  remaining  nodes  will 
be  partitioned  in  the  network  design  stage.  This  approach  is  less 
computationally  costly  than  completing  the  partitioning  in  the 
selection  process,  but  is  also  less  effective  as  the  selected  sites 
are  .o  longer  the  center  of  mass  of  the  final  partitions. 

In  the  above  procedure,  the  parameter  Z may  be  interpreted  as  a 
capacity  constraint  for  a backbone  switch.  This  constraint  in- 
troduces additional  complexities  to  the  procedure.  By  not  per- 
mitting the  size  Z to  be  exceeded,  the  process  of  developing  clus- 
ters may  stop  prematurely  for  lack  of  feasible  mergers.  In  this 
case,  simply  the  N largest  clusters  are  used.  Furthermore,  mergers 
may  occur  over  extraordinary  distances  because  of  feasibility  issues 
This  is  certainly  not  desirable,  and  so  a parameter  of  maximum 
distance  between  mergeable  nodes  is  introduced.  Finally,  if  modular 
capacity  switches  are  available,  then  several  clusters  in  the  immedi 
ate  vacinity  of  one  another  are  probably  better  served  by  one  large 
switch.  Consequently,  a parameter  is  available  to  define  a minimal 
separation  between  clusters.  Clusters  closer  than  this  minimum 
distance  are  combined  to  be  served  by  one  switch.  The  algorithm  can 
now  be  defined  as  follows: 


Input 


Number  of  backbone  switch  locations  to  be  selected 


Cost  weighing  factor  of  node  i 


Maximum  size  of  cluster  expressed  in  terms  of  V 


Percent  of  Z over  which  cluster  growth  is  stopped 
and  cluster  center  of  mass  selected  as  backbone  node 
location. 


Minimum  radius  of  cluster 


Maximum  distance  over  which  clusters  can  be  merged 
(expressed  as  a ^or.Scaiit  times  the  average  nearest 
Keighbor  distance  of  <j>)  . 


2.  Merge  the  two  neavest  nodes  and  replace  by  one  node  at 
their  center  of  maa*  with  their  combined  cost  weighing  factors 
m.  and  attribute  vectors  V.;  provided  that 
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J 


a.  The  combined  attribute  vectors  do  not  violate  Z. 

b.  The  clusters  are  not  further  apart  than  a. 

If  no  nodes  can  be  merged  go  to  5. 

3.  If  the  new  cluster  has  size  greater  than  aZ,  its  center 

of  mass  is  added  to  the  list  of  selected  backbone  nodes  fig, 

and  the  cluster  is  deleted  from  the  problem;  provided  that 

the  new  cluster  is  not  within  r of  any  other  member  of  the 

list  £3  ; otherwise,  the  new  cluster  and  its  closest  other 
s 

member  of  the  list  are  merged  and  returned  to  the  list  ftg. 

4.  If  the  number  of  elements  in  ft  is  less  than  N,  go 

s 

to  2;  else  go  to  5. 


5.  Output  as  the  selected  nodes  the  members  of  ftg.  If 
there  are  less  than  N members  select  the  remaining  locations 
on  the  basis  of  the  largest  remaining  clusters. 

A detailed  description  of  the  algorithm  is  provided  in  the  next 
section. 

2.4.4  Description  of  Algorithm 

In  this  section  a detailed  description  of  the  Heuristic  Cluster- 
ing Algorithm  is  presented. 

Definitions 

N = Number  of  backbone  switch  locations  to  be  selected. 

* = {i|i  * - set  of  all  nodes. 


i 
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Set  of  nodes  associated  with  i in  t 


Coordinates  of  node  i in  I 


Cost  of  weight  factor  of  node  i in  * 


Z - Maximum  size  of  cluster,  expressed  in  terms  of  w 


Percent  of  Z over  which  cluster  growth  is  stopped  and 
the  cluster  center  of  mass  is  used  to  select  backbone 
node  location. 


r - Minimum  radius  of  cluster 


a - Constant  factor  used  to  define  maximum  distance  over 
which  clusters  can  be  merged. 


Functions 
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Clustering  Algorithm  for  Switch  Site  Selection  (CASSS) 
Step  1:  Initialization: 

A.  DO  = {d.  (j)  | d.  (j)  = min  d.  U)  , i = 1,2,... 

1 i 1Cq  i 

iyi 

g(wi  'W*)-Z 


B.  For  i = 1,2, ... ,1,  let 
= (i}. 


C.  For  each  d^(j)  eDO,  let 
ni  = j. 


D-  davg  = W" 


E.  d = a x d 

max  avg 


P.  D = ta1(n1)|a1(ni)  < dmax}. 


G. 

Step  2; 


o.  * >• 


Merge : 


A. 


dJl(n)l)  = min  d^n^ 


B.  Form  a new  node  k with 

ralVVXn. 

Y i i 

Ak  m +m  ' 

* n. 


Yk  - 


mlYt+mnYn 
l l 

■vv 


5 


i 


Y.  mjYj  * m*  Yk 

3 mj  + mk 

Pj  * PjUV 

Wj  «-  g (Wj  ,wR) . 

Else  if  w.  >aZ,  then  fl  = fl  U(k>. 

Iv  s s 

B.  D1  » D - (di(ni)  |nie{*,na>}> . 

C.  *2  * { i [ ie  * , n^U,  n^}}. 

D.  If  k { n , then:  4»2«-*2U{k}, 

S 

U {k}. 


— H — M — . — H I ■ I H — I 


At  the  end  of  Step  5,  we  have  a set  of  backbone  switch  sites 
and  associated  nodes.  We  may  now  apply  appropriate  techniques  for 
the  evaluation  of  backbone  line  costs  and  local  access  costs,  as 
discussed  in  Sections  2.5  (approximations)  and  2.7  (detailed  designs). 
It  should  be  noted  that  the  node  associations  with  the  backbone 
switches  do  not  form  a complete  partition  of  the  nodes.  Such  a com- 
plete partitioning  could  be  accomplished  as  an  insertion  of  the  fol- 
lowing between  B and  C of  Step  5: 

BC.  If  then: 

a. 

b. 

c. 


Select  i such  that  m.  = max  m., 

1 j e 4>  3 

Find  j such  that  d.  (j)  = min  d.  (A) , 

1 Aefi  x 

m.X.  + m.X. 

v J J ^ ^ 


■i + “i 


m.Y.  + m.Yj 
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d. 

pj 

- PiUPj, 

e. 

w . 
3 

g(wi,w;.) 

f . 

m. 

mi  + m j , 

g.  4>  = 4>  - { i}  I 


h . go  to  BC . 

The  above  "post-processing"  would  provide  a partition  of  the 
nodes  in  which  each  backbone  switch  site  minimizes  local  access  cost 
for  its  partition  element.  However,  the  "post-processing"  above 
does  not  guarantee  feasibility  relative  to  the  capacity  constraints. 
Further  refinements  by  assigning  each  node  individually  could  elim- 
inate this  difficulty.  Thus,  the  algorithm  can  be  continually  de- 
veloped and  refined,  with  associated  cost  increases  and  performance 
improvements. 

2.4.5  Complexity  Analysis  of  the  Algorithm 

In  this  section,  we  analyze  the  computational  complexity  of 
the  algorithm  as  described  in  Section  2.4.4.  In  the  Initialization 
Step  Part  A is  an  0(NT  ) search  of  nearest  neighbors  for  all  nodes 
(where  NT  is  the  total  number  of  nodes).  The  remaining  parts  of 
the  steps  are  all  0(NT)  or  less.  Thus,  the  time  computational  com- 
plexity of  Step  1 is: 

tx  = 0(Nt2)  + 0(Nt). 


/ 


2.42 


Network  Analysis  Corporation 


Steps  2,  3 and  4 are  iterated  until  N merged  nodes  have  attribute 
vectors  exceeding  oZ,  or  until  no  further  mergers  are  possible.  As 
a worst  case,  they  would  be  exercised  (N^-N)  times.  In  Step  2, 
the  only  computationally  significant  processing  is  the  selection 
of  the  minimum  in  Part  A.  If  the  values  of  d^(.)  are  maintained 
in  a heap  throughout  the  procedure,  then  the  processing  in  Part  A 
is  0(1);  however,  at  some  point  the  heap  must  be  restored.  Assigning 
this  restoral  processing  as  part  of  A,  we  get: 


t2  = 0(Nt  log  Nt) 


In  Step  3,  Part  A requires  a comparison  against  the  elements 
of  n , which  are  at  most  N-l  in  number.  The  remaining  steps  de- 

S 

termine 


1.  The  new  nearest  neighbor  for  the  new  node, and 


2.  The  new  nearest  neighbors  for  the  nodes  which  had 
as  nearest  neighbors  one  of  the  two  merged  nodes. 


In  the  implementation,  it  is  actually  not  necessary  to  process  2. 
in  every  iteration.  Rather,  when  a value  di ( j ) is  selected  in  Step 
2 from  the  top  of  the  heap,  then  node  j can  be  checked  on  whether  it  is 
a valid  neighbor.  If  not,  then  and  only  then  will  a new  nearest 
neighbor  be  searched.  With  this  approach,  the  processing  in  Parts 
B through  H of  Step  3 can  be  viewed  as  the  task  of  reaching  for  a 
new  nearest  neighbor  and  restoring  the  heap  for  an  average  of  0(NT-N) 
nodes.  With  even  a simple-minded  search  method,  the  nearest 
neighbor  can  be  found  in  0(NT)  operations.  Thus,  the  computational 
complexity  of  Step  3 is 


t3  < 0(Mt->  + 0 (N) . 
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The  computational  complexity  for  Step  4 is  0(1).  In  order 
to  define  the  computational  complexity  of  Step  5,  let: 

Ng  = Number  of  backbone  candidate  sites  (= | fi | ) » 


[ 


Np  = Number  of  partition  elements  when  the  algorithm 
stops  (N<Np<NT) . 

Then, 

t5  = 0(NxNfi)  + 0 (Np)  + 0 (Nfix  N log  Np) 

Summarizing  the  above  results,  the  total  computational  com- 
plexity can  be  expressed  as 

5 2 

t = ill  fci  = °<nt  > + 0(V 

+ 0(Nt  log  Nt) 

+ 0(Nt2)  + 0 (N) 

+ 0(NxNb)  + 0 (Np)  + 0 (NgxN  log  Np) . 

Hence, 


2 

t = 0(Nt  ) + 0(Nt  log  Nt)  + lower  order  terms. 

The  biggest  burden  of  computational  complexity  is  the  searching 

. 2 

for  nearest  neighbors  in  Steps  1 and  3,  giving  the  0(NT  ) term.  With 
reasonable  accuracy,  this  complexity  can  be  reduced  to  0(HxNT),  where 
H is  the  average  number  of  nodes  per  box  defined  by  a rectangular 
grid  over  the  space  of  interest.  The  accuracy  will  depend  on  the 
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fineness  of  the  grid  as  will  the  value  of  H.  The  assignment  of 
nodes  to  a particular  box  can  be  made  in  linear  time  (0(NT)).  in 
general,  the  complexity  reduction  achieved  in  this  manner  with 
acceptable  accuracy  leaves  0(NT  log  NT)  and  0(HxNt)  the  dominant 
terms . Thus , 


t « 0(N^,  log  N,p)  + 0(Hxn,j,)  + lower  order  terms. 


assuming  the  use  of  a grid  approximation  technique  in  nearest 
neighbor  searches.  Note  that  during  Step  3,  because  of  the  cluster- 
ing of  nodes,  we  may  have  to  modify  the  grids  from  time  to  time. 
This  grid  approximation  technique  can  also  be  used  for  the  Add  Algo- 
rithm and  the  Drop  Algorithm  described  in  the  subsequent  sections. 
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2.5  THE  ADD  APPROACH 

2.5.1  Basic  Assumptions  and  Procedure  Development 

We  now  consider  the  formulation  of  a heuristic  solution  pro- 
cedure for  the  BNSP  based  on  ideas  similar  to  the  Add  Algorithm 
given  for  the  Warehouse  Location  Problem  (WLP)  in  Section  2.  The 
BNSP  is  inherently  more  complex  than  the  WLP,  thus  we  have  to 
propose  a number  of  assumptions  to  make  the  problem  tractible. 

First  we  consider  the  concept  of  the  central  facility,  which 
is  fundamental  to  the  add  approach.  Since  a distributed  network 
does  not  possess  a center,  we  have  to  create  a center  artificially 
and  give  it  a different  interpretation.  Suppose  we  have  chosen 
backbone  nodes  C^,...,  already  (k  may  be  0,  in  which  case  no 
backbone  node  has  been  chosen  yet)  . If  all  terminals  are  assigned 
to  one  of  the  C.'s,  i=l,...,k,  then  we  are  done.  Suppose,  on  the 
other  hand,  that  not  all  the  terminals  are  assigned.  Let, 


<t0  = Set  of  terminals  not  yet  assigned. 

If  we  select  only  one  more  backbone  node,  then  a reasonable  choice 
for  this  node,  Cg,  is  the  traffic-weighted  center  of  mass  of  the 
nodes  in  4>g.  That  is,  if 

tT  = Total  traffic  of  a terminal  T, 


= The 

Cartesian 

coordinate 

for  T, 

<w 

= The 

Cartesian 

coordinate 

for  CQ, 

then 
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x0  = d t X )/l  t , 

Te$0  Te$0  1 


■0  = d V )/^  t 


Te$Q  - * Te4>g 


(2.3) 


Let  Q be  a backbone  candidate  not  yet  used.  If  we  use  Q as  the 
(k+1)  th  backbone  node,  then  we  would  have  to  take  some  of  the 
nodes  in  (nodes  not  yet  assigned)  away  from  Cq,  and  assign 
them  to  Q.  Thus,  the  basic  tradeoff  is  between  homing  nodes  to 
Q,  or  homing  nodes  to  CQ.  Hence  CQ  serves  as  the  central  facility 
in  the  add  approach. 


Assumption  1:  Suppose  we  have  already  chosen  backbone  nodes  C^, 
...,Ck,  and  is  the  set  of  terminals  not  yet  assigned,  then,  CQ, 
the  weighted  center  of  mass  of  the  nodes  in  <I>0  as  given  by  (2.3), 
can  be  considered  as  the  central  facility  in  the  selection  of  the 
(k+1)  backbone  node  in  the  add  approach. 

Note  that  CQ  is  updated  each  time  we  add  a backbone  node. 

We  now  consider  the  cost  tradeoff  of  assigning  a terminal  to 
Q versus  assigning  it  to  Cq.  First  we  examine  how  we  treat  this 
problem  in  the  WLP  setting.  Suppose  T is  a terminal  (end  office) 
not  yet  assigned  to  any  of  the  C^'s,  i=l,...,k.  The  saving  of 
assigning  T to  Q is  given  by  (see  Figure  2.7): 

St(Q)  = CTxt(rxd(T,C0)-[CTxtTxd(T,Q)+CQxtTxd(Q,C0)], 

where 

tT  = Quantity  of  goods  required  at  T, 

CT  = Unit  transportation  cost  per  unit  distance 
per  unit  traffic  from  CQ  to  T, 
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CQ  = Unit  transportation  cost  per  unit  distance 
per  unit  traffic  from  Cq  to  Q, 

and  d(A,B)  = Euclidean  distance  between  two  points  A,B. 

Next  we  consider  this  problem  in  the  BNSP  setting.  Suppose 
T is  a terminal  not  yet  assigned,  and  T'  is  a terminal  assigned 
to  C^,  ic{0,...,k}.  Let  t,p  T,  be  the  requirement  between  T and 
T'.  Then  by  the  same  principle  as  in  the  WLP,  the  saving,  for 
the  (T,T ' ) pair,  of  assigning  T to  Q is  given  by  (see  Figure  2.8) 


ST,T(Q) 


t_  T, [CTxd (T,Cn)  + (cost  of  transmitting  a unit 


traffic  from  Cq  to  C^) ] 


- t_  _,[CTxd(T,Q)  + (cost  of  transmitting  a unit 
' traffic  from  Q to  C . ) ] - 

1 (2.4) 

Now,  the  topological  design  problem  for  packet-switched  net- 
works is  in  general  quite  an  involved  task,  and  without  an  actual 
design,  there  is  no  way  to  know  precisely  which  path  would  transmit 
the  messages  between  two  given  points  on  the  network.  Thus  the  cost 
of  transmitting  a unit  traffic,  say,  from  CQ  to  C.^ , as  required  in 
Formula  (2.4),  cannot  be  calculated  precisely.  However,  for  our 
purpose,  approximate  values  for  these  unit  costs  suffice.  Consider 
Figure  2.9.  Let  A,  B be  two  points  on  a P/S  network.  There  may 
or  may  not  be  a direct  arc  connecting  A,  B.  However,  the  length 
of  the  transmission  paths  from  A to  B (for  example,  A-D-E-B  and/or 
A-C-B  in  Figure  2.9)  usually  should  be  somewhat  dependent  upon 
d (A,B) . Here  we  propose  to  estimate  the  length  of  the  average 
transmission  path  between  A,  B by 

pxd (A,B) , 

where  p is  a proportionality  factor  determined  by  the  traffic  matrix 
of  the  P/S  network.  In  fact,  for  ease  of  computation,  we  are  making 
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a stronger  assumption.  We  suppose  that  this  proportionality 
factor  depends  only  on  the  number  of  nodes  in  the  P/S  network. 

Our  second  assumption  is  then  as  follows. 

Assumption  2 : Let  H be  a minimum  cost  P/S  network  topology  with 

N nodes  and  given  traffic  matrix.  Then  the  unit  line  cost  of 
transmitting  packets  between  two  points  A,B  on  II  can  be  approxi- 
mated by 

C (N)  x d(A,B)  , 

where  C(i)  is  a proportionality  factor  dependent  only  upon  N. 

Further,  we  suppose  that  C(*)  is  insensitive  to  slight  variations 
in  N. 

With  Assumption  2,  we  can  now  evaluate  Formula  (2.4).  Returning 
to  the  cost  tradeoff  examination,  suppose  <b*  is  a subset  of 
(hence  the  nodes  in  4*  are  tentatively  assigned  to  CQ.)  Let 

t^*  = Total  traffic  of  nodes  in  4>*, 

and  for  i=0,l,...,k. 


t^*,i  = Total  traffic  between  nodes  assigned  to 

(except  those  in  $*)  and  nodes  in  4>*. 

By  Assumption  2,  if  we  assign  the  nodes  in  4>*  to  Cp,  the  backbone 
line  cost  is  approximately 


CBX  1=1  t**,i  x d(Co,Ci), 


(2.5) 
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where  Cg  denotes  the  unit  backbone  line  cost  factor  as  defined  in 
Assumption  2.  (See  Figure  2.10)  Thus  the  backbone  line  saving  of 
assigning  nodes  in  $*  to  Q is 


However,  these  calculations  are  quite  cumbersome.  Actually,  we 

are  only  interested  in  the  weighted  aggregations,  BLCQ  and  BLC^, 

as  given  by  (2.5)  and  (2.6).  Thus  a rough  estimate  for  the  t»A  .'s 

w* , i 

suffices.  A reasonable  estimate  for  BLCQ  and  BLCQ  is  to  make  the 
t$*,i's  Pr°P°rtional  to  the  total  traffic  at  the  (^'s.  Even  though 
this  estimate  may  be  less  valid  for  the  individual  t**  .'s,  the 

▼ r 1 

effect  of  the  weighted  summation  as  appeared  in  (2.5)  and  (2.6) 
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should  tend  to  cancel  out  the  individual  deviations  and 
reasonable  estimate.  Thus,  let 


Total  traffic  of  all  nodes  (sum  of  transmit  and 
receive  traffic), and 


Total 


Total  traffic  of  nodes  (except  those  in  4>  ) 
assigned  to  C.. 


Then  substituting  the  t 


we  have 
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Then  if  is  the  set  of  nodes  which  can  be  assigned  profitably  to 
Q,  we  can  estimate  tg  by 

k 

t0  w ^otal  “ |=1  ti  ~ fcAV*  (2.11) 

From  (2.10)  and  (2.11),  the  backbone  line  saving  for  unit  band- 
width of  using  Q as  a new  backbone  node  is  then  approximately 

C k 

BLS(Q)  « — r ^ x l t.  x [d(Cn,C.)  - d (Q  ,C  . ) ] (2.12) 

^Total  i=0  1 01  1 

Assumption  3 : Suppose  we  have  already  chosen  backbone  nodes  , . . . , 

and  CQ  is  the  weighted  center  of  mass  of  nodes  not  yet  assigned 
as  given  by  Formula  (2.3).  If  Q is  a backbone  candidate  not  yet 
chosen,  then  the  unit  backbone  line  saving  of  using  Q as  the  (k+l)~th 
backbone  node  is  given  approximately  by  (2.12). 

Note  that  Assumptions  1 and  2 are  essential  for  the  adoption 
of  the  add  approach  in  BNSP.  Assumption  3 is  not  essential,  but  is 
introduced  to  simplify  the  computation. 

We  now  evaluate  the  total  (local  and  backbone,  fixed  and  line) 
saving  associated  with  using  a backbone  candidate  Q.  Let  T be  a 
terminal  not  yet  assigned  to  any  of  the  Ci's,  i=l,...,k.  By  For- 
mulae^.4)  and  (2.12),  the  (local  access  and  backbone)  line  saving 
of  assigning  T to  Q,  instead  of  CQ,  is  approximately 

ST(Q)  = x [d(T,CQ)  - d (T,Q)  ] + tT  x BLS(Q),  (2.13) 

where  BLS(Q)  is  given  by  (2.12),  and 

tT  * Total  traffic  of  node  T . 

= Mileage  cost  of  the  local  access  line  connecting 
T to  the  backbone. 
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(Note  that  we  could  also  use  tT  x CT  in  place  of  KT  in  (2.13). 
However,  since  KT  is  a better  indicator  of  the  actual  local  ac- 
cess cost  involved,  and  also  it  can  be  derived  quite  easily,  we 
prefer  to  use  KT  over  tT  x CT  in  calculating  the  local  access 
savings.)  By  Formula  (2.13),  we  can  determine  for  each  Q,  which 
terminal  has  the  greatest  saving,  ST(Q),  which  terminal  has  the 
second  greatest  saving,  etc.  Thus,  we  assign  terminals  to  Q 
until  the  constraints  on  Q are  violated,  or  until  no  more  saving 
can  be  achieved.  The  total  saving  for  Q is  then 


where  C„  is  the  fixed  cost  of  setting  up  a backbone  node.  The  Q 
with  the  greatest  saving  is  selected  as  the  (k+1)  backbone  node 


A Modularized  Backbone  Node  Model 


The  add  approach  as  formulated  in  Section  2.5.1  is  quite  gen- 
eral and  flexible.  The  various  backbone  node  capacity  constraints 
important  to  the  particular  application  at  hand  can  be  incorporated 
in  the  terminal  to  backbone  candidate  assignment  process.  However, 
there  are  a number  of  situations  in  which  the  standard  add  approach 
tends  to  give  unsatisfactory  results.  First,  when  the  capacity  con- 
straints are  loose  and  the  backbone  node  fixed  cost  is  small  relative 
to  the  capacity  and  the  line  costs,  the  add  approach  tends  to  pro- 
duce fewer  than  the  optimum  number  of  backbone  nodes.  For  example, 
suppose  both  the  terminal  location  and  the  terminal  traffic  distri- 
bution are  fairly  uniform.  Then  the  first  CQ  location  would  be  some- 
where near  the  center.  If  there  is  no  capacity  limitation,  the  first 
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backbone  node  picked  may  take  away  close  to  one  half  of  the 
terminals.  (See  Figure  2.11.)  This  is  clearly  undesirable.  Sec- 
ond, in  many  cases,  the  backbone  node  cost  is  not  fixed,  but  rather 
varies  with  the  capacity,  as  depicted  in  Figure  2.12.  This  situa- 
tion can  be  handled  by  taking  an  average  value  as  the  backbone 
node  fixed  cost.  But  similar  to  the  first  situation,  this  ap- 
proach would  tend  to  discourage  the  selection  of  low  capacity 
backbone  nodes.  An  alternative  is  to  calculate  the  fixed  cost  for 
each  backbone  candidate  after  we  have  made  the  tentative  assign- 
ment in  each  iteration.  However,  this  approach  is  too  time  con- 
suming, and  to  a lesser  extent,  still  has  the  same  drawback  as 
the  first  approach.  (Note  that  the  backbone  node  cost  is  actually 
the  difference  between  the  cost  of  setting  up  a new  backbone  node 
to  handle  a certain  amount  of  traffic,  and  the  cost  of  handling  the 
same  traffic  at  CQ.) 

We  propose  here  a modification  to  the  basic  add  approach  to 
handle  the  above  situations.  We  consider  each  backbone  node,  in- 
stead of  as  an  aggregated  whole,  as  composed  of  several  basic 
modules,  each  with  a fixed  capacity.  Thus,  each  backbone  module 
has  a fixed  traffic  handling  capacity,  but  each  backbone  node  es- 
sentially has  unlimited  capacity.  This  is  in  fact  a better  as- 
sumption if  we  use  the  HSIMP's  at  the  backbone  node  locations. 
During  each  iteration  of  the  add  procedure,  instead  of  adding 
one  more  backbone  node,  we  are  adding  only  one  more  backbone 
module  to  the  network.  The  intention  is  that  with  this  modu- 
larization in  capacity,  the  situation  depicted  in  Figure  2.11 
will  not  occur. 

Now  we  consider  the  backbone  nodal  cost  model.  We  probably 
do  not  want  to  charge  all  the  nodal  cost  to  the  first  module  placed 
at  a backbone  node  location.  However,  we  do  want  to  charge  more 
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for  the  first  module  placed  at  a location  than  for  the  second 
module  placed  at  the  same  location,  etc.  Let 


BBC  = Total  backbone  nodal  cost 


module  at  a backbone 


Then  we  require 


(usually  BBC  > AC. ) 


The  cost  model  we  use  in  this  study  is 


(A)  and  (C)  are  automatically  satisfied 


Then  by  (2.15)  and  (2.16) 


Note  that  when  PF  is  large 


Of  course,  this  model  is  only  one  of  the  many  possible  models 
we  can  use.  However,  it  does  satisfy  criteria  (A) -(C),  and  models 
the  curve  depicted  in  Figure  2.12  fairly  well.  Moreover,  by  varying 
the  value  assigned  to  PF , we  can  obtain  different  backbone  node 
cost  compositions. 


Description  of  the  Algorithm 


Below  we  outline  the  Add  Algorithm  on  the  modularized  capacity 
model  as  developed  in  2.5.1  and  2.5.2.  A simplified  overall  flowchart 
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is  also  given  in  Figure  (2.13). 
Notations 


* 

n 

nbc 

Bi 
♦ (Q) 


IT 

ACX 

AC(Q) 


CP  (Q) 
PF 


Set  of  all  terminals . 

Set  of  all ^backbone  candidates . 

Number  of  backbone  candidates. 

i th  backbone  candidate  in  i=l,...,NB(,. 

Set  of  terminals  assigned  to  a backbone  node  Q. 

Set  of  terminals  not  yet  assigned  to  a 
backbone  node* 

Traffic  weighted  center  of  mass  of  nodes  in  4>0. 
Set  of  backbone  nodes  chosen. 

Cost  of  placing  the  first  module  at  a backbone 
node  location  as  defined  in  Formula  (2.16). 

Backbone  node  incremental  cost  (for  adding  one 
more  module)  for  node  Q. 

Total  backbone  node  cost  for  node  Q. 

Proportion  factor  as  defined  in  Formulae  (2.15) 
and  (2.16). 
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Maximum  backbone  node  fixed  cost  (as  defined 
in  Section  2.5.2)  . 

The  backbone  candidate  with  the  maximum  saving 
in  an  iteration. 


The  set  of  terminals  assigned  to  Q* 


The  saving  associated  with  Q*. 


Modular  Capacity  Add  Algorithm 


Step  1:  Initialization: 


A.  For  each  Qeft, 


CP  (Q)  + tclt 


AC(Q)  «-  ^ xBBC , 


b.  n +■  0, 


% * *• 


Step  2 : Find  the  location  of  Cn  by  Formula  (2.3) 


Step  3:  Initialize  Iteration: 


j *■  0 . 


SA  «-  0 . 
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Step  4:  Savin?  Calculation  Iteration 


j - j+1. 


If  j > N_c , go  to  Step  6, 


Step  5;  Update  Saving : 


A.  Find  assignment,  ¥ , and  saving,  S,  for  B.  by  Formulae 
(2.13)  and  (2.14),  based  on  the  greedy  criterion  and  module 
capacity  constraint. 


B.  If  S > SA,  then 


SA  ♦ S, 


**  «-  V, 


Q*  ♦ By 


C.  Go  to  Step  4. 


Step  6 : Update  Backbone  Node  Information : 


A.  If  SA  < 0,  go  to  Step  7. 


B.  *o  * *0  ' **? 


♦ (Q*)  ♦ *(Q*)  U **; 


AC (Q* ) ♦ AC (Q* ) x pj. 


CP (Q*)  ♦ CP (Q*)  + AC (Q*) ; 


n «■  n u (Q* } . 


- - - — 
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A.  Find  Q*eft  which  is  nearest  to  C 


At  the  end  of  Step  7 , we  have  a suboptimal  set  of  backbone 
nodes  and  associated  terminals.  We  may  now  apply  appropriate  tech' 
niques  for  the  evaluation  of  backbone  line  costs  and  local  access 
costs,  as  discussed  in  Section  2.3. 


Complexity  Analysis  of  the  Algorithm 


In  this  section  we  analyze  the  computational  complexity  of  the 
modularized  add  approach  as  outlined  in  2.5.3. 


Total  number  of  terminals 


Total  number  of  backbone  candidates 


Total  number  of  backbone  nodes  selected 


Average  number  of  terminals  assigned  to  a 
backbone  module  • 
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and 

Nm  = Average  number  of  modules  used  at  a backbone 
node  location. 

The  notations  defined  in  2.5.3  will  also  be  used  here  without  further 
explanation. 

Notice  that  from  the  definitions,  we  can  conclude  that 
total  number  of  modules  placed  on  the 

backbone  network  « Ng  x NM,  (2.19) 


and 

Nt»  Ng  x Nm  x Nav.  (2.20) 

We  analyze  the  time  spent  for  each  step  of  the  Add  Algorithm. 
Notice  that  Steps  1 and  7 are  each  applied  only  once,  and  Steps  2-6 
are  applied  iteratively.  Moreover,  Steps  2,  3 and  6 are  each  applied 
roughly  N x N times,  while  Steps  4 and  5 are  each  applied  roughly 
Ns  x Nm  x NfiC  times.  For  i=l,...,7,  let 

t^j  = Total  time  spent  for  all  applications  of 
step  i of  the  algorithm. 


Then  trivially, 

7 

total  time  spent  - l t... 

i=l 
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It  can  be  easily  concluded  that 


‘(1)  = 0(nbc>  + 0(nt>' 


t (3)  - »l»s  X V' 


°{HS  X Nm  X Hbc), 


‘ (7)  < 0(nbc>  + °<V- 

Also,  each  application  of  Step  6 involves  roughly  o (NAV)  operations, 

because  the  size  of  4>*  can  be  estimated  by  N.„.  Thus 

A V 

t(6)  ® 0(NS  X NM  X NAV>' 


t(6)  » omT) 


by  Formula  (2.20) 


It  remains  to  estimate  t^j  and  t(5)‘ 


First,  we  consider 


t (2 ) • By  Formula  (2.3), 


x0  " (I  XT  V7  E fcT ' 

w Te$0  Te4>0 


Y0  <1  YT  fcT}/iL  tT)- 
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Note  that  XQ  and  YQ  can  be  calculated  incrementally,  Thus,  let 

t = I , 


= I xt 

Te4>0 


Then  XQ  =tX/t  and  YQ  = tY/t.  After  each  iteration  of  the  procedure 
the  new  values  for  t,  tx  and  tY  can  be  obtained  by 


t - t - I tT, 


tx  * t -l  Xjptjp, 


Te<I>* 


t - l 


Te<J>* 


Note  that  each  terminal  will  be  in  exactly  one  <J>*  throughout  the 
whole  procedure.  Thus  after  the  initial  calculation,  the  total 
update  time  for  t,  tX  and  tY  through  all  the  iterations  is  only 
0(Nt).  The  initial  calculation  for  t,  tx  and  tY  also  takes  time 
0(Nt).  Hence,  we  conclude  that 


Network  Anelytit  Corporation 


Now  we  consider  t^.  First,  note  that  each  application  of 
part  (b)  of  Step  5 involves  roughly  NAV  operations.  Next,  we 
remark  that  part  (a)  of  Step  5 involves  three  main  tasks: 

1.  Calculation  of  the  unit  backbone  line  saving 
for  Bj  by  Formula  (2.12): 


BLS (B_. ) 


T — — l t.  x [d(C  ,C.)  - d (B  . ,C  . ) ] , 
'"Total  i=0  1 u 1 3 1 


where  C^,...,C^  are  the  backbone  nodes  already 
selected,  and  t^  is  the  traffic  assigned  to  C . , 
i=l, . . . ,k; 

2.  Calculation  of  the  line  saving  of  homing 
terminals  T to  B^  by  Formula  (2.13)  for  all 
T in  4>q: 


ST(Bj)  = KT[d(T,C0)  - d (T  ,B  j ) 1 + tT  X BLS(B.); 

3.  Extracting  successively  the  largest  S^B^J's 

until  we  reach  the  module  capacity  constraint; 

this  corresponds  to  extracting  roughly  the  NftV 

largest  elements  from  {S_ (B . ) }_  . 

1 ] I £<Pg  • 


Consider  Task  1.  Notice  that 


BLS (B  . ) = - 

J 


Total 


1 Lo  - wv  - ?=1  v(vci>! 


and  each  time  a module  is  added  to  the  backbone,  l tid(CQ,Ci)  and 


rw 

tg  (Bj ,Cg)  need  only  be  calculated  once, while  £ tid(Bj,C^) 
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need  only  be  updated  incrementally  by  a constant  number,  say  C, 
of  operations  for  each  . Hence  the  average  number  of  operations 
involved  in  Task  1,  when  there  are  k backbone  nodes  selected,  is 
roughly 


nbc  + k 


N 


+ C < C+2 


BC 


Next  consider  Task  2.  The  calculation  of  each  (B ^ ) involves 

only  a constant  number  of  operations.  Hence  Task  2 takes  time 
at  most  N,j,.  Finally  we  consider  Task  3.  The  problem  of  selecting 
the  m largest  elements  from  a set  of  n real  numbers  can  be  done 
in  (n-1)  + (m-1)  flog  n]  comparisons  by  setting  up  a heap  with 
the  larger  elements  closer  to  the  top,  and  successively  extracting 
the  largest  element  from  the  top;  the  heap  can  be  first  built  in 
n-1  comparisons  and  then  it  can  be  rebuilt  after  each  extraction 

in  at  most  Tlog  ril  comparisons.  (See  Spira  [SPIRA,  1973],  and  Aho, 
Hopcraft  and  Ullman  [AHO,  1974].)  Hence  Task  3 can  be  done  in 

Nt  + Nav  flog  Nt"| operations.  We  thus  conclude  that  Step  5 can 
be  done  in  time 


(5)  - 


Ns  x NM  x N^  (Nav  + C+2  + NT  + NT  + Nav  flog  N^.1) 

NS  X NM  X NBC  (2NT  + NAV  llQ9  Nt1> 

Ns  X nm  X nbc  X 2nt  + nbc  x nt  x [ 109  nt1 

(by  Formula  (2.20)) 

Nt  x Nbc  (2xNg  x Nm  + flog  nJ  ). 
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In  summary,  the  total  time  spent  in  the  modularized  Add 
Algorithm  is 

7 

total  time  = \ t . . . 

i-1  U) 

= 0[Nt  x Nbc  (NsxNm  + [log  N?1 ) ] . (2.21) 

Note  that  we  can  usually  assume  that 
Nt  » Nbc  » Ns  » Nm. 

It  is  interesting  to  note  that  the  standard  add  approach 
(without  the  modularization)  requires  time 

°[Nt  x Nbc  (Ns  + flog  NT1)]. 

There  are  always  ways  to  reduce  the  time  complexity  of  the 
add  appreach.  For  example,  if  we  partition  the  total  area  into  a 
suitable  number  of  subregions,  and  only  assign  terminals  to  back- 
bone candidates  in  the  same  region,  then  the  factor  NT  in  Formula 
(2.21)  can  be  reduced  quite  appreciably.  However,  from  the  ex- 
perimental results  shown  in  Section  2.7,  the  running  time  of  the 
modularized  add  approach  seems  quite  satisfactory  for  networks  of 
the  size  considered  in  this  study. 

We  now  consider  the  storage  requirement.  In  addition  to  the 
storage  required  for  the  terminal  and  backbone  node  database 
(coordinates,  traffic,  etc.)  we  need  roughly  NT  memory  locations 
to  store  the  list  Nftv  locations  for  the  list  $*,2  Ng(,  locations 
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. wrri 


for  AC  ( • ) and  CP  ( • ) , and  Ng  locations 

bone  nodes.  In  addition,  we  need 
k 

S t.d(B.,C.)'s  and  N„  locations  to 
i=X  1 3 1 

quired  in  Task  1 of  Step  5(A)),  NT  locations  to  store  the  heap  of 
S,j,  (B  j ) ' s , (required  in  Tasks  2 and  3 of  Step  5(A)  and  a small  num- 
ber of  locations  for  other  quantities  like  t,  tX,  BLS,  etc.  Hence, 
the  additional  core  is  roughly 


to  store  the  selected  back- 
locations  to  store  the 

store  the  tQd (B^ ,CQ) ' s , (re- 


x NT  + 


X NBC  + 


nav  + 


N 


S‘ 
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2.6  DROP  APPROACH 


2.6.1  Introduction  and  Outline 


Similar  to  the  add  approach,  the  drop  approach  is  also  a stand- 
ard heuristic  solution  procedure  for  the  warehouse  location  problem 
(WLP) , and  thus  can  be  adopted  to  the  BNSP.  In  a sense,  the  Drop 
Approach  tackles  the  problem  in  a totally  opposite  direction  from 
the  add  approach.  In  the  context  of  the  BNSP,  initially  every 
backbone  candidate  is  used  as  a node  in  the  backbone  network.  An 
obvious  terminal  to  backbone  node  assignment  is  that  of  assigning 
each  terminal  to  its  nearest  backbone  candidate.  Then,  at  each 
iteration  of  the  procedure  we  calculate  (estimate)  the  saving  that 
results  from  eliminating  each  candidate  and  reassigning  the  associated 
terminals  to  other  suitable  candidates.  The  candidate  with  the  larg- 
est positive  saving  is  selected  for  elimination.  We  then  reassign 
the  terminals,  and  repeat  the  process.  If  no  positive  saving  can  be 
achieved  by  dropping  backbone  candidates,  the  procedure  is  stopped 
and  the^  remaining  candidates  represent  the  suboptimum  solution  for 
the  backbone  network  design. 

There  are  three  distinct  components  for  the  saving  associated 
with  dropping  a backbone  candidate:  local  access  line  saving,  switch- 
ing node  saving  and  the  backbone  line  saving.  When  dropping  a node 
from  the  backbone  network  we  observe  the  following  facts:  (1)  the 
backbone  node  saving  is  positive  and  corresponds  roughly  to  the 
switch  installation  cost;  (2)  the  backbone  network  usually  has  fewer 
links  and  carries  less  traffic,  thus  leading  to  backbone  line  savings; 
(3)  the  reassigned  terminals  are  farther  away  from  the  backbone  net- 
work, and  usually  local  traffic  is  increased  (see  Table  2.14  of  Sec- 
tion 2.7),  so  that  the  local  access  line  savings  are  usually  negative. 
The  tradeoff  is,  therefore,  between  the  combined  switch  and  backbone 
line  savings  and  the  local  access  cost  increase.  The  backbone  node 
fixed  change  can  be  regarded  as  constant  throughout  the  procedure. 
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The  local  access  line  saving  can  be  calculated  exactly  if  so  de- 
sired. The  backbone  line  saving  is  calculated  using  approximate 
techniques.  Both  of  these  results  depend  to  a certain  extent  on 
the  method  used  for  terminal  reassignment.  Many  different  tech- 
niques can  be  used  in  terminal  reassignment,  local  access  saving 
calculation  and  backbone  line  saving  calculation.  The  choice  de- 
pends on  the  tradeoffs  between  such  factors  as  time,  storage  re- 
quirement and  accuracy.  Several  alternatives  are  discussed  in 
Section  2.2. 

A flow  chart  of  the  drop  procedure  is  giver  in  Figure  2.14. 
2.6.2  Saving  Calculation  Alternatives 

In  this  section,  we  let 

Nt  = Number  of  terminals. 

NBC  = Number  of  backbone  candidates. 

N..^  = Average  number  of  terminals  assigned 

*"  til 

to  a remaining  candidate  in  the  i 
iteration  of  the  drop  procedure. 

We  have  for  i = 1 , 2 , . . . , 

nav  " V(nbc  - 1 + 11  • 

A.  Terminal  Reassignment 


When  we  consider  the  effect  of  eliminating  a backbone  candi- 
date, say  Q,  during  an  iteration  of  the  drop  procedure  (Box  4 in 
Figure  2.14),  or  when  we  have  already  chosen  a backbone  candidate 
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FIGURE  2.11:  OUTLINE  OF  THE  DROP  PROCEDURE 


BACKBONE  CANDIDATES. 

NUMBER  OF  BACKBONE  CANDIDATES. 


SA  - maximum  saving  of  dropping 

CANDIDATE 

B - CANDIDATE  CORRESPONDING  TO 
J BEST  SAVING. 
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Q to  be  eliminated  at  the  end  of  an  iteration  (Box  12  in  Figure  2.14), 
we  have  to  (tentatively  or  permanently)  reassign  the  terminals  ori- 
ginally assigned  to  Q to  other  remaining  candidates.  The  obvious 
solution  to  this  problem  is,  for  each  terminal  T originally  assigned 
to  Q,  to  find  the  second  nearest  candidate,  say  P.  The  local  access 
line  saving  of  reassigning  T is  then 


LSt(Q)  = KT[d(T,  Q)  - d (T,  P)  ] , 


(2.23) 


where 


Kt  = Mileage  cost  of  local  access  line  from  T. 


d(*,»)  = Distance  between  two  points. 


Note  that  LSt(Q)  is  always  negative. 

Consider  how  much  time  we  have  to  spend  for  the  terminal  re- 
assignment calculations  in  the  i th  iteration  of  the  drop  procedure. 
Since  each  terminal  is  assigned  to  exactly  one  of  the  candidates 
remaining  after  the  (i-1)  ^ iteration,  the  tentative  reassignment 
is  done  NT  times.  At  the  end  of  the  iteration,  one  candidate  is 

1 i 

selected  to  be  dropped.  There  are  roughly  NAV  terminals  assigned 
to  this  node.  Thus  the  permanent  reassignment  is  done  roughly 
Na^  times.  In  a straightforward  fashion,  each  reassignment  re- 
quires a time  proportional  to  N _.  The  total  time  required  in  the 

— dL 

i iteration  is  then 


°t«T  + NAV>  * NBCI- 
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We  can  improve  the  computation  effort  somewhat.  Suppose  we 
first  find  the  k(say,  k=5)  nearest  candidates  for  each  terminal. 
This  would  take  0(k  x NT  x NBC)  comparisons.  Then#  each  search  for 
the  second  nearest  candidate  requires  essentially  only  one  time 
unit.  Thus,  the  total  time  required  is: 


0 [ (k+2)  x N_  x N^] 


However,  note  that  this  time-saving  is  done  at  the  expense  of 
additional  storage.  For  each  terminal  T,  we  have  to  store  its  k 
nearest  backbone  candidates.  We  also  need  backward  pointers  from 
the  candidates  to  the  terminals.  The  extra  space  required  is  thus 


We  can  improve  this  storage  requirement,  but  at  the  expense 
of  accuracy.  Instead  of  storing  the  k nearest  candidates  informa 
tion  for  each  terminal,  we  store  this  information  for  each  candi- 
date . Thus , only 


extra  space  is  needed.  Now  for  a terminal  T assigned  to  a candi- 
date Q,  instead  of  searching  for  its  second  nearest  candidate,  we 
search  for  the  candidate  nearest  to  T among  the  k nearest  candidates 
to  Q.  Thus,  the  search  time  is  roughly  k units.  Note  that  the 
candidate  selected  may  be  different  from  the  second  nearest  candi- 
date to  T.  However,  we  expect  this  to  happen  only  rarely.  The 
total  (tentative  and  permanent)  reassignment  time  required  is  then 
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il 


Tra  * °<k  * *>BC  * "BC1  + * 0!<NT  + NAV>  * kl 

x-nbc 

= 0 (k  x Nb^)  + O (k  x NT  x Nk) 


0(k  x Nt  x Nbc) , 


(2.26) 


which  is  comparable  to  (2.25) 


B.  Local  Access  Saving 


Suppose  Q is  a candidate  considered  for  elimination,  T is  a 
terminal  assigned  to  Q,  and  P is  the  second  nearest  candidate  to 
T.  Then  by  Formula  (2.23)  derived  in  Section  2. 6. 2. A,  the  local  ac- 
cess saving  of  reassigning  T is  given  by 


LSt(Q)  = Kt  x [d (T,  Q)  - d (T,  P) ] , 


(2.23) 


where  KT  = unit  cost  of  local  access  line  from  T.  The  total  local 
access  saving  of  reassigning  T is  given  by 


LS(Q) 


T assigned  to  Q 


LSt(Q) . 


(2.27) 


According  to  Formula  (2.27),  the  LS(Q)'s  are  evaluated  by  cal- 
culating the  LSt(Q)'s.  Since  at  any/f^tage  of  the  procedure,  each 
terminal  is  assigned  to  a unique  candidate,  so  in  the  i_th  itera- 
tion of  the  drop  procedure.  Formula  (2.23)  is  applied  at  most  twice 
for  each  terminal  T (once  during  the  tentative  reassignment  phase 
and  once  during  the  permanent  reassignment  phase) . It  follows 
then  that  the  total  time  spent  in  the  local  access  saving  calcula- 
tion is 


T * E 2N_ 

IOC 


0(Nt  X Nbc) 


(2.28) 


— 
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C.  Backbone  Line  Saving 

Now  we  evaluate  the  backbone  line  saving  resulting  from  the 
elimination  of  a backbone  candidate.  To  improve  computational  ef- 
ficiency we  will  use  the  backbone  line  cost  estimate  developed  in 
Section  2.5. 

For  each  backbone  candidate  Q,  let 


BLC(Q)  = Total  cost  of  backbone  lines  used  to 

accommodate  the  traffic  to  and  from  Q, 

ULC(Q)  = Backbone  line  cost  for  a unit  traffic 
to  or  from  Q using  the  backbone , 


tg  = Total  traffic  to  and  from  Q. 


Also,  let 


-Total  = Total  traffic  to  and  from  all  the  terminals. 


CR  = Backbone  line  cost  proportion  factor  as  de- 
fined in  Assumption  2 of  Section  2.5.1. 

Then  by  Assumptions  2 and  3 of  Section  2.5.1, 


ULC(Q)  = x l t d(R,Q) 

'"Total  K 

Rebackbone  node  set 


(2.29) 


BLC  (Q)  = tQ  • ULC  (Q)  . 


(2.30) 


(Compare  to  Formula  (2.12)  in  Section  2.5.1) 
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Suppose  we  are  in  the  i * iteration  of  the  drop  procedure# 
and  we  consider  the  effect  of  eliminating  a backbone  candidate  Q. 


= {p|p  is  a backbone  node  and  some  terminal's  assigned 


to  Q are  reassigned  to  P}, 


and  for  each  Pe4>Q,  let 

Atp  = The  amount  of  traffic  (tentatively)  rehomed  to 
P from  Q. 


Then  it  can  be  shown  that,  by  using  Formulae  (2.29)  and  (2.30),  the 
backbone  line  saving  resulting  from  eliminating  Q is: 


BLS(Q)  = 


(BLC(Q)  - l AtpULC (P) ] 


C 

+1/2  x”  • l At°At°[d(P,Q)  + d (R,Q) 

^otal  _ _ . P R 
P,Re$Q 


- d (R,P) ] 


(2.31) 


The  above  formula  is  rather  cumbersome.  However,  the  BI£(Q)'s 
and  the  ULC(P)'s  need  only  be  incremented  at  each  iteration,  so  the 
number  of  additions  and  multiplications  involved  in  (2.31)  is 
actually  proportional  to  |$q|  . Moreover,  note  that  the  |$q|'s 
are  usually  quite  small,  and  for  all  practical  purposes  we  can  sup- 
pose that  they  are  bounded  by  a small  constant,  h (say,  h=10) . 

2 

Hence  the  number  of  operations  is  bounded  by  h . It  follows  then 
that  in  the  i”th  iteration,  the  time  spent  for  calculating  the 
BLS (Q) ' s is  of  order 

0 [h2  x (nbc  - i+1)]  = 0(h2  x Nbc). 
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Now  we  consider  the  time  needed  for  updating  the  BLC (Q) ' s and  the 

ULC (Q) ' s.  Since  this  is  done  only  once  in  each  iteration,  and 

roughly  only  h of  the  Q's  have  incremental  At0's,  the  time  required 

* 

in  the  i iteration  is  approximately 


Hence,  the  total  time  required  in  the  drop  procedure  for  the  back 
bone  saving  calculation  is 
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Straight-forward 

terminal 

reassignment 

find  k nearest 
candidates  for 
each  terminal 

find  k nearest 
candidates  for 
each  candidate 

Terminal 

Time 

0(Nt  x N^,) 

0(k  x NT  x Nb) 

0(k  x NT  x Nb) 

Extra 

Space 

+0 

+ 2 x k x Nt 

+ 2 x k x Nb 

Local  access 
saving 

Time 

0(Nt  x Nbc) 

Backbone  line 
saving 

Time 

0(h2  * M2.) 

Time 

0(Nt  x 4,) 

0(k  x Nt  x Nq 
+ h2  x N^J 

0(k  x Nt  x Nb 

+ h2  * 4* 

Overall 

Extra 

Space 

+0 

+ 2 x k x N_ 
T 

+ 2 x k x Nn 

D 

Accuracy 

more  accurate 

more  accurate 

less  accurate 

TABLE  2.1:  COMPARISON  OF  THE  CALCULATION  ALTERNATIVES 


N„ 


Number  of  terminals* 


N 


BC 


Number  of  backbone  candidates. 


k 

h 


Number  of  nearest  candidates  selected 
(for  a terminal  or  candidate) . 


Average  number  of  candidates  involved 
in  a reassignment. 


(Note  that  the  extra  space  is  relative  to  the  first  alternative) 


\ 
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2.7  EXPERIMENTAL  RESULTS 
2.7.1  Test  Environment 

A.  Solution  Procedures  Tested 

The  following  four  BNSP  solution  procedures  have  been  tested 
in  this  study. 

1.  Modularized  add  procedure  (as  discussed  in 
Section  2.5). 

2.  Cluster  procedure  (as  discussed  in  Section  2.4). 

3.  Manual  selection. 

4.  Random  generation. 


The  first  two  procedures  have  been  experimented  quite  thoroughly 
as  they  provide  most  of  the  results  in  this  section.  The  manual 
selection  method  and  the  random  generation  method  are  used  to  gen- 
erate a few  designs  for  comparative  purpose. 

B.  System  Models 

We  have  experimented  the  BNSP  procedures  on  the  following  two 
system  models: 

1.  AUTODIN  II  System:  This  is  the  integrated  communication 

system  of  the  ADP  systems  administered  by  the  Defense 
Communication  Agency.  The  AUTODIN  II  system  size  is  as 
follows : 
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Total  number  of 

: user  locations 

= 

300 

number  of 

host  computer 

= 

86 

number  of 

concentrators 

= 

26 

number  of 

TDMX * s 

= 

85 

number  of 

TCU’s 

= 

11 

number  of 

isolated  terminals 

= 

101 

Total  traffic 

= 

1.26  Megabits/sec 

Figure  2.15  is 

a plot  of  the  AUTODIN 

II  system. 

2.  A Randomly  Generated  Communication  (RGC)  System: 
This  is  a system  generated  specifically  for  the  pur- 
pose of  studying  the  BNSP  procedures.  The  terminal 
location  and  traffic  volume  are  generated  in  proportion 
to  population  distribution  of  the  United  States.  The 
traffic  and  location  pattern  of  this  system  is  thus  in 
general  more  uniform  than  that  of  the  AUTODIN  II  system. 
The  RGC  system  size  is  as  follows: 


Total  number  of  user  locations 

= 

450 

number  of 

host  computers 

= 

100 

number  of 

concen  t ra  tor  s 

= 

179 

number  of 

TDMX ' s 

= 

45 

number  of 

TCU's 

= 

45 

number  of 

isolated  terminals 

= 

81 

Total  traffic 

= 

12.0 

Figure  2.16  is 

a plot  of  the  RGC 

system. 

Tables  2.2  and  2.3 

list  the  facility  capacities 

the  two  systems.  The  local  access  facilities  are  the  same  for  the 
two  systems,  and  are  listed  in  Table  2.2.  More  powerful  switch 
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TABLE  2.2:  LOCAL  ACCESS  FACILITY  UNIT  COST 

(for  both  AUTODIN  II  and  RGC) 

2.89 
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SOFTWARE  DEVELOPMENT  ONE-TIME 
FIXED  COST 


Switch  node  fixed  cost 


Trunk  line 

^ Host  or  Concentrator  line 

O TCU  line 

u 

w MUX  line 

< 2.4  KB/S  terminal  line 

4.8  KB/S  terminal  line 
S3  9.6  KB/S  terminal  line 

H 

50.  KB/S  terminal  line 

Number  of  host  lines 
Number  of  terminal  lines 
Number  of  terminals 
Total  host  traffic 
Total  terminal  traffic 
Total  traffic 


AUTODIN  II 

RGC 

$59, 800/mo. 

$59, 800/mo. 

10,600 

42,400 

87 

932 

87 

87 

87 

87 

12 

12 

12 

12 

62 

62 

75 

75 

87 

87 

25 

100 

100 

400 

200 

800 

500  KB/S 

2,000  KB/S 

480  KB/S 

1,920  KB/S 

1,750  KB/S 

7,000  KB/S 

TABLE  2.3:  BACKBONE  SWITCH  CAPACITY  AND  UNIT  COST 
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nodes  and  communication  links  on  the  other  hand,  are  used  on  the 
backbone  S/F  network  for  the  RGC  system,  since  the  traffic  volume  of 
the  RGC  system  is  approximately  10  times  larger  than  that  of  AUTODIN 
II  system.  The  backbone  facility  capacities  and  costs  are  listed  in 
Table  2.3. 


C.  Backbone  Candidate  Sets 


In  this  study,  the  following  two  types  of  backbone  candidate 
sets  have  been  used. 

1.  The  set  of  AUTOVON  sites: 

This  is  the  set  of  approximately  60  AUTOVON 
locations.  AUTOVON  locations  are  fairly  well  dis- 
tributed geographically,  and  are  a subset  of  the 
AUTODIN  II  system  locations.  Figure  2.17  shows  a 
plot  of  the  AUTOVON  locations. 

2.  A subset  of  all  the  terminal  sites: 

Without  a prespecified  set  of  backbone  candidates, 
a natural  choice  for  the  candidate  set  is  the  total 
set  of  terminal  (i.e.,  host  and  co- located  terminal) 
locations.  With  the  cluster  approach,  we  can  con- 
sider all  the  terminals  as  candidates,  because  the 
running  time  of  the  cluster  approach  is  not  affected 
by  the  size  of  the  candidate  set.  With  the  add  ap- 
proach, whose  running  time  is  proportional  to  the 
size  of  the  candidate  set,  we  can  choose  the  k ter- 
minal sites  with  the  largest  amount  of  traffic,  for 
some  appropriate  k.  In  this  study,  we  have  generated 
backbone  selections  on  the  RGC  system  with  k « 100. 
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FIGURE  2.17:  AUTOVON  LOCATIONS 
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D.  Solution  Generation  Process 

Here  we  describe  briefly  how  we  obtain  the  various  data  points 
in  the  experiments. 

Since  the  total  cost  curve  (relative  to  the  number  of  back- 
bone nodes)  is  approximately  U-shaped  (as  discussed  in  Section  7. 2. A), 
if  we  are  merely  interested  in  obtaining  the  (heuristically)  optimum 
solution,  only  a few  designs  need  be  generated.  However,  since  we 
are  interested  in  fully  understanding  the  relation  between  the 
backbone  node  selection  and  other  factors  of  the  overall  network 
design,  we  have  generated  a full  range  of  designs  with  different 
number  of  backbone  nodes. 

With  the  cluster  approach,  the  method  for  obtaining  designs 
with  different  number  of  backbone  nodes  is  quite  simple.  The 
three  parameters  involved  are: 

NB  = Number  of  backbone  nodes  desired, 

* 

P * Proportionality  factor,  0 < P < 1, 

and 

ttotal  = »ystem  traffic. 

During  the  process,  the  clusters  will  be  merged  until  their  total 
traffic  exceeds  (ttotal/NB)  x P.  At  the  end  of  the  process,  the 

NB  largest  clusters  are  selected,  and  backbone  nodes  are  placed  in 
their  centers  of  mass.  Hence  by  simply  varying  NB,  we  can  obtain 
designs  with  different  number  of  backbone  nodes.  In  all  of  our 
cluster  experiments  P has  been  set  to  .5. 
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With  the  add  approach,  the  parameters  involved  ares 
MC  = Module  capacity, 

BBC  = Backbone  switch  cost, 

PF  = Module  cost  proportional  factor, 

RUTPN  = Non-direct  routing  penalty, 

and 


LUTIL  = Average  line  utilization. 

Actually,  MC  is  a collection  of  constraints  such  as  maximum  num- 
ber of  Host  line  allowed,  maximum  number  of  terminal  lines  allowed 
maximum  Host  traffic  allowed,  etc.  During  the  experiment,  we  have 
assumed  that  the  constraints  are  obtained  by  scaling,  by  a fixed 
coefficient,  the  switch  capacity  constraints  listed  in  Table  2.3. 
Hence  we  can  speak  of  the  module  capacity  as  being  1/2  the  switch 
node  capacity,  etc. 

From  Formula  (2.15),  the  cost  of  placing  the  k module  at  a 
node  location  is 

1 (1_  pf  l"l)  + pf  1 x BBC  if  k * 1, 

/ 1 x BBC  otherwise. 


\ 


In  our  experiment  with  the  add  procedure,  MC  has  been  varied 
from  2 (leading  to  designs  with  very  few  backbone  nodes)  to  1/8 
of  the  switch  node  capacity.  BBN  was  varied  accordingly.  For  given 
values  of  MC  and  BBC,  we  guess  (or  derive  from  previous  experiments) 
the  number  of  backbone  nodes  that  will  be  generated,  and  thus  set 
the  appropriate  values  for  RUTPN  and  LUTIL.  With  MC,  BBC,  RUTPN  and 
LUTIL  all  fixed,  we  vary  PF  over  its  range,  and  obtain  different 
designs  with  (usually)  slightly  different  numbers  of  backbone  nodes. 


2.7.2 


Tables  2.4  and  2.5  summarize  the  results  of  ADD  and  CLUSTER 
algorithms  for  the  AUTODIN  II  system  and  the  RGC  system  respectively. 
Total  network  cost  is  reported  as  a function  of  backbone  switches 
used  in  the  design.  Some  of  the  topologies  are  shown  in  Figures 
2.18  through  2.26.  Solid  lines  are  local  access  lines  and  dotted 
lines  are  backbone  trunks. 

The  remainder  of  this  section  contains  a discussion  of  the 
experimental  results. 

A.  Total  Cost  vs.  Number  of  Backbone  Nodes 


From  Tables  2.4  and  2.5  we  can  observe  the  following: 

1.  If  for  any  particular  design  procedure  (add  or 
cluster) , we  plot  the  total  cost  of  the  best  design 
versus  number  of  backbone  nodes,  then  these  points 
very  closely  represent  a convex  (U-shaped)  curve 
with  a unique  minimum  point. 

This  phenomenon  can  be  explained  by  examining 
the  cost  composition  of  the  system.  As  discussed 
in  Section  2.6.1  (see  also  Tables  2.8  through  2.12)  when 
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SOLUTION 

PROCEDURE 

ADD 

CANDIDATE  SITE 
OPTION 

AUTOVON 

CLUSTER 


AUTOVON 


CLUSTER 


ALL  TERMINALS 


8 8 


w 10 
w 

8 u 


S 

x 14 


fa  16 

O 


TABLE  2.4:  TOTAL  COST  (K$/M0)  OF  DESIGNS  PRODUCED*  AUTOD 


NOTE:  If  a procedure  produces  more  than  one  design  with  the  same 

number  of  backbone  nodes,  the  one  with  the  lowest  total 
cost  is  recorded. 
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TABLE  2.5:  TOTAL  COST  (K$/MO)  OF  DESIGNS  PRODUCED:  RGC  SYSTEM 


NOTE:  If  a procedure  produces  more  than  one  design  with  the 

same  number  of  backbone  nodes,  the  one  with  the  lowest 
total  cost  is  recorded. 
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AUTODIN  II  SYSTEM  WITH  11  SWITCHES.  ADD  ALGORI 


FIGURE  2.20;  AUTODIN  II  SYSTEM  WITH  19  SWITCHES.  ADD  ALGORITHM 


RGC  SYSTEM  WITH 


the  number  of  backbone  nodes  for  the  system  in- 
creases, both  the  backbone  line  cost  and  the  back- 
bone nodal  cost  usually  increase,  while  the  local 
access  cost  usually  decreases.  The  composition  of 
the  three  components  thus  produces  a nearly  convex 
cost  curve. 

should  caution  that  we  do  not  imply  that 
the  total  cost  curve  i£  convex.  Hence,  we  can  not 
obtain  the  heuristic  optimum  design  by  simply  search- 
ing for  a local  minimum  point.  However,  in  the 
special  case  when  the  total  system  design  is  modified 
incrementally  (for  example,  by  adding  backbone  nodes 
one  at  a time  as  in  the  add  approach) , it  can  be 
shown  that  the  total  cost  curve  is  convex  [TANG,  1974] . 
In  fact,  the  stopping  condition  in  both  the  add  approach 
and  the  drop  approach  is  based  on  this  result. 


2.  The  optimum  solution  is  in  the  range  of  9 - 12 
nodes.  (Except  for  the  case  of  cluster  solution  ap- 
plied to  the  RGC  system  with  all  terminals  used  as 
backbone  candidates;  in  such  case  the  optimum  is  20. 

The  total  design  cost  trend  is  somewhat  puzzling  in 
this  case.) 

It  is  a useful  fact  that  within  the  same  sys- 
tem (AUTODIN  II  or  RGC)  different  selection  methods  all 
yield  optimum  designs  with  the  number  of  backbone 
nodes  in  one  small  range.  However,  it  mu»t  be  co- 
incidental that  in  our  study  the  optimum  number  of 
backbone  nodes  for  both  the  AUTODIN  II  system  and  the 
RGC  system  fall  within  the  same  range.  In  fact. 
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the  total  traffic  in  the  RGC  system  is  10  times 
that  of  the  AUTODIN  II  system,  but  the  capacity  of  the 
backbone  switch  node  and  the  backbone  line  are  also 
much  higher.  If  the  same  switching  and  line  facil- 
ities as  the  AUTODIN  II  system  are  employed  in  the  RGC 
system,  we  would  expect  the  optimum  number  of  back- 
bone nodes  for  the  RGC  system  to  be  much  higher  than 
that  for  the  AUTODIN  II  system. 

3.  The  curves  of  total  cost  versus  number  of  back- 
bone nodes  are  very  flat.  In  the  range  considered 
here  (5  - 20  nodes) , the  difference  between  the  min- 
imum point  and  the  maximum  point  usually  amounts  to 
only  10  percent  of  the  total  cost.  Moreover,  the 
curves  are  flatter  around  the  optimum  points.  These 
results  suggest  that,  with  a reasonable  amount  of 
computation  time,  we  can  usually  obtain  a fairly  good 
design  with  total  cost  within  a few  percent  of  the 
optimum  design. 


B.  Comparison  of  the  Solution  Procedures  in  Terms 
of  the  Designs  Produced 

Prom  Tables  2.4  - 2.7,  we  can  make  the  following  conclusions. 

1.  The  designs  produced  by  the  modularized  add  ap- 
proach are  uniformly  the  best  of  the  four  procedures 
tested.  This  should  come  as  no  surprise  because,  of 
the  four  procedures,  the  add  approach  achieves  a fairly 
accurate  cost  tradeoff  between  the  different  ingre- 
dients of  the  total  cost. 


i 
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(a)  Randomly  Selected  Designs 


TOTAL  COST 


1011  K$/mo. 
1111 


(b)  Manually  Selected  Designs 


RUN  TOTAL  COST 

1 897  K$/Mo. 

2 767 
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(a)  Randomly  Selected  Designs 


TOTAL  COST 


5954  K$/mo. 


5368 


ft 


5104 


5434 


5287 


(b)  Manually  Selected  Designs 


TOTAL  COST 


4935  K$/mo. 
5003 


TABLE  2,7:  MANUALLY  AND  RANDOMLY  SELECTED  DESIGNS;  RGC  SYSTEM 

11  BACKBONE  NODES  SELECTED 

(CANDIDATE  SET  IS  THE  SET  OF  AUTOVON  LOCATIONS) 


2.110 


2.  With  the  knowledge  of  the  possible  range  of  the 
optimum  number  of  backbone  nodes  and  a good  familiarity 

■I 

j with  the  system  model  under  consideration,  a designer 

will  be  able  to  select  backbone  locations  nearly  as 
good  as  the  best  produced  by  an  automated  procedure. 

This  is  illustrated  in  Tables  2.6(b)  and  2.7(b).  We 
have  much  more  experience  with  the  AUTODIN  II  system  than 
with  the  RGC  system,  and  are  thus  able  to  produce  a 
comparably  much  better  manual  design  with  the  AUTODIN  II 
system.  These  results  also  suggest  that  the  best  design 
would  be  obtained  by  manually  perturbing  the  optimum 
locations  produced  by  automated  procedures. 

3.  The  designs  produced  by  random  selection  are  un- 
iformly the  worse.  This  confirms  the  intuition  that 
selection  with  some  kind  of  intelligence  (as  in  the 
manual  selection  case)  is  always  better  than  no  in- 
telligence at  all. 


4.  The  total  costs  of  the  randomly  produced  de- 
signs for  the  AUTODIN  II  system  range  from  $857K  to 

1$1111K,  while  the  cost  for  the  best  design  we  found 
is  $742K.  The  total  cost  of  the  randomly  produced 
designs  for  the  RGC  system  range  from  $5104K  to 
$5954K,  while  the  cost  for  the  best  design  we 
found  is  $4395K.  (All  of  the  above  designs  are 
obtained  with  the  AUTOVON  locations  as  the  back- 
bone candidates.)  These  figures  give  an  indi- 
cation of  the  range  of  the  total  cost  of  a pos- 


! 

sible  desian. 
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C.  Cost  Composition 

From  Tables  2.8  - 2.12,  we  can  observe  the  following. 

1.  Among  the  three  main  components  of  the  total 
cost,  the  local  access  cost  is  clearly  dominant 
over  the  backbone  switch  node  cost  and  the  backbone 
line  cost.  This  fact  suggests  that  in  trying  to  de- 
sign a near-optimum  configuration,  we  should  spend 
more  effort  in  examining  the  local  cost  tradeoff 
than  the  other  two  components. 

2.  When  the  number  of  backbone  nodes  in  the  design 
increase,  the  local  access  cost  usually  descreases, 
while  both  the  backbone  switch  cost  and  the  backbone 
line  cost  increase.  This  can  be  explained  intuitively 
that  as  the  number  of  nodes  in  the  backbone  increases, 
usually: 

a.  The  terminals  are  closer  to  the  backbone, 

b.  The  local  traffic  descreases  (and  thus  the 
backbone  traffic  increases) , because  terminals 
are  more  and  more  likely  to  be  assigned  to  dif- 
ferent backbone  nodes  (see  also  Table  2.13), 
and 

c.  The  number  of  links  in  the  backbone  in- 
creases. 

3.  Compared  to  the  cluster  approach,  the  add  approach 
tends  to  produce  designs  with  lower  local  access  cost 
and  comparable,  or  slightly  higher  backbone  line  cost. 
This  result  can  be  explained  by  the  following  reasonings 
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NUMBER  OF 
BACKBONE 
NODES 

LOCAL  ACCESS 
COST 

BACKBONE 

SWITCH 

COST 

BACKBONE 
LINE  COST 

TOTAL  COST 

4 

537 

118 

143 

798 

5 

500 

129 

147 

775 

6 

470 

139 

163 

772 

7 

441 

150 

166 

757 

8 

429 

160 

164 

753 

9 

406 

171 

173 

750 

10 

- 

- 

- 

- 

11 

360 

192 

190 

742 

12 

357 

203 

196 

756 

13 

- 

- 

- 

- 

14 

362 

224 

205 

791 

15 

329 

245 

223 

797 

16 

- 

- 

- 

17 

- 

- 

- 

- 

18 

326 

266 

235 

828 

19 

325 

277 

240 

843 

TABLE  2.8:  COST  COMPOSITION 

Model:  AUTODIN  II 


Solution  Procedure:  Add  approach  with  AUTOVON  locations  as  candidates 

Unit:  K$/mo. 
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NUMBER  OF 
BACKBONE 
NODES 

LOCAL  ACCESS 
COST 

BACKBONE 

SWITCH 

COST 

BACKBONE 
LINE  COST 

TOTAL  COST 

4 

5 

511 

129 

156 

795 

6 

491 

139 

165 

795 

7 

459 

150 

170 

779 

8 

455 

160 

176 

792 

9 

449 

171 

185 

805 

10 

400 

182 

191 

773 

11 

392 

192 

198 

782 

12 

389 

203 

204 

796 

13 

381 

213 

20S 

804 

14 

364 

224 

213 

801 

15 

358 

235 

220 

812 

16 

352 

245 

226 

823 

17 

347 

256 

233 

836 

18 

- 

- 

- 

- 

19 

338 

277 

246 

860 

TABLE  2.9:  COST  COMPOSITION 

Model:  AUTODIN  II 

Solution  Procedure:  Cluster  approach  with  AUTOVON  locations  as  candidates 

Unit:  K$/mo . 
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NUMBER  OF 
BACKBONE 
NODES 

LOCAL  ACCESS 
COST 

BACKBONE 

SWITCH 

COST 

BXCK60NE 

LINE 

COST 

6 

3,575 

349 

856 

7 

. 3,292 

391 

926 

8 

3,071 

434 

957 

9 

3,008 

476 

962 

10 

2,909 

518 

969 

11 

2,853 

561 

982 

12 

2,841 

603 

993 

13 

2,806 

646 

1,019 

14 

- 

- 

- 

15 

2,764 

730 

1,030 

16 

2,735 

773 

1,031 

17 

2,619 

815 

1,072 

18 

2,626 

858 

1,067 

24 

2,536 

1,112 

1,139 

26 

2,530 

1,197 

1,153 

TABLE  2.10:  COST  COMPOSITION 


TOTAL  COST 

4,780 

4,610 

4,462 

4,446 

4,396 

4,395 

4,437 

4,471 

4,525 

4,539 

4,504 

4,551 

4,787 

4,880 


NUMBER  OF 
BACKBONE 
NODES 

LOCAL  ACCESS 
COST 

BACKBONE 

SWITCH 

COST 

BACKBONE 

LINE 

COST 

TOTAL  COST 

8 

2,695 

434 

1,021 

4,150 

10 

2,489 

518 

1,022 

4,029 

12 

2,327 

603 

1,034 

3,964 

14 

2,263 

688 

1,066 

4,017 

17 

2,167 

815 

1,109 

4,091 

21 

2,048 

985 

1,142 

4,175 

TABLE  2.11:  COST  COMPOSITION 


Model:  RGC 

Solution  Procedure:  Add  Approach  with  100  best  terminals  as  candidates 

Unit:  K $/mo. 
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NUMBER  OF 
BACKBONE 


LOCAL  ACCESS 


BACKBONE 

SWITCH 


COST 

8 

3,048 

9 

2,874 

10 

2,969 

11 

2,801 

12 

2,765 

16 

2,528 

20 

2,087 

25 

1,888 

BACKBONE 

SWITCH 

COST 

BACKBONE 

LINE 

COST 

TOTAL  COST 

434 

909 

4,380 

476 

964 

4,314 

518 

951 

4,438 

561 

960 

4,322 

603 

973 

4,340 

645 

1,059 

4,232 

942 

1,144 

4,173 

1,154 

1,172 

4,214 

TABLE  2.12:  COST  COMPOSITION 

Model:  RGC 

Solution  Procedure:  Cluster  Approach  with  all  terminals  as  candidates 

Unit:  K $/mo. 
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NUMBER^v^YSTEM 

OF 

BACKBONE 

NODES  \ 

AUTODIN 

r 11 

, 

RGC 

4 

415 

mm 

5 

356 

mBM 

6 

313 

7 

256 

WSSsm 

8 

254 

BBS 

9 

174 

1,415 

10 

- 

1,073 

11 

159 

1,164 

12 

159 

1,045 

13 

- 

930 

14 

142 

- 

15 

- 

893 

16 

128 

825 

17 

- 

808 

18 

120 

930 

19 

120 

- 

24 

- 

702 

26 

702 
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a.  Under  the  most  favorable  circumstance,  the 
cluster  approach  would  assign  the  terminals  in 
each  cluster  to  the  center  of  mass  of  that  cluster. 
However,  in  practice,  when  the  cluster  procedure 
terminates,  only  a portion  of  the  cluster  centers 
are  selected  to  be  the  backbone  nodes,  and  many 
terminals  are  thus  assigned  to  the  centers  of 
other  clusters.  Hence  the  cluster  centers  selected 
at  the  termination  of  the  process  usually  do  not 
correspond  to  the  centers  of  their  assigned  ter- 
minals. This  fact  leads  to  the  local  access  cost 
increase. 

b.  The  add  approach  explicitly  considers  all 
three  components  of  the  total  cost,  while  the 
cluster  approach  essentially  considers  only  the 
local  access  cost.  In  not  considering  the  back- 
bone line  cost,  the  cluster  approach  can  be  re- 
garded as  neutral  to  the  backbone  line  cost 
tradeoff,  and  hence  does  not  take  advantage,  of 
the  fact  that  the  backbone  lines  are  comparably 
cheap. 

This  result  justifies  the  observation  made  in  2.7.2.B  that  the  add 
approach  consistently  produces  lower  cost  designs  than  the  cluster 
approach. 

D.  Effect  of  Available  Backbone  Candidate  Locations 


Prom  Tables  2.4  and  2.5,  we  can  observe  that  the  designs  gen- 
erated by  using  all  terminals,  or  the  100  best  terminals,  as  the 
backbone  candidates,  have  lower  overall  cost  than  the  designs  gen- 
erated by  using  the  AUTOVON  locations  as  the  backbone  candidates. 
This  can  be  explained  as  follows. 
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1.  The  set  of  all  terminals,  or  the  set  of  100 
best  terminals,  give  a better  choice  of  the  ter- 
minal cluster  centers  thi.n  the  set  of  AUTOVON 
locations,  because  they  are  better  correlated  with 
terminal  traffic  and  geographical  distribution. 

2.  The  set  of  all  terminals,  or  the  set  of  100 
best  terminals,  contains  more  elements  than  the 
set  of  AUTOVON  locations,  and  hence  in  a sense 
offers  more  choice  in  selecting  the  backbone 
nodes . 


It  is  interesting  to  note  that  on  the  AUTODIN  II  system,  the 
design  generated  by  using  all  terminals  as  backbone  candidates  are 
only  slightly  better  than  the  corresponding  designs  generated  by 
using  the  AUTOVON  locations  as  backbone  candidates,  while  on  the  RGC 
system,  the  differences  are  much  more  pronounced.  This  can  be 
explained  by  the  fact  that  all  the  AUTOVON  locations  correspond  to 
terminal  sites  in  the  AUTODIN  II  system,  thus  they  provide  a better 
match  for  the  AUTODIN  I I requirements  than  for  the  RGC  requirements. 

E.  Sensitivity  of  Optimum  Design  to  Unit  Component 


Cost  Variations 

In  this  section  we  consider  the  effect  of  unit  component  cost 
changes  on  the  optimum  design.  The, three  main  components  of  the 
total  cost  are  the  local  access  cost,  the  switch  cost  and  the  back- 
bone cost,  and  each  of  these  again  has  two  components.  Thus  the 
six  components  of  the  network  cost  are  as  follows: 

LL  * Local  access  mileage  cost,  the  part  of  the  local 
access  cost  that  is  associated  with  the  line 
mileages. 
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LH  » Local  access  hardware  cost,  the  part  of  the  local 
access  cost  that  is  associated  with  the  hardware 
at  the  terminals,  the  modems,  the  interfaces. 

SB  * Switch  basic  cost,  the  part  of  the  switch  cost 
that  is  associated  with  mainframe  fabrication, 
installation,  operation,  maintenance,  etc. 

Typically,  this  is  a fixed  cost  per  node. 

SM  = Switch  modular  cost,  the  part  of  backbone 

switch  cost  that  is  associated  with  switch- 
to-switch  and  switch-to-terminal  interfaces 
and  buffer  and  processor  resources  to  handle 
them.  Typically  this  cost  differs  from  node- 
to-node,  but  the  sum  over  all  nodes  is  invar- 
iant with  respect  to  the  number  of  nodes. 

BL  * Backbone  line  mileage  cost,  the  part  of  the 
backbone  line  cost  that  is  associated  with 
the  line  mileages. 

BH  = Backbone  line  hardware  cost,  the  part  of  the 
backbone  line  cost  that  is  associated  with 
the  modems,  etc.  ' 

In  Tables  2.14  and  2.15  we  list  the  cost  of  the  six  components 
Dr  the  AUTODIN  II  designs  and  the  RGC  designs.  In  Figures  2.27  and 
.28  we  plot  the  six  component  costs  against  the  number  of  back- 
>ne  nodes.  We  find  the  interesting  property  that  in  the  range 
! are  concerned  with  (5  — 20  nodes) , five  of  the  six  cost  curves 
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I It 

] 

* 

1 . 


MN 

LOCAL 

ACCESS 

BACKBONE 

SWITCH 

BACKBONE  LINE 

MILEAGE 

COST 

HARDWARE 

COST 

BASIC 

COST 

MODULAR 

COST 

MILEAGE 

COST 

MODEM 

COST 

4 

447 

90 

42 

76 

128 

15 

5 

410 

90 

53 

76 

129 

18 

6 

380 

90 

64 

76 

141 

22 

7 

351 

90 

74 

76 

141 

25 

8 

338 

90 

85 

76 

136 

28 

9 

319 

87 

95 

76 

142 

31 

10 

- 

- 

- 

- 

- 

- 

11 

275 

85 

117 

76 

153 

37 

12 

272 

85 

127 

76 

156 

40 

13 

- 

- 

- 

- 

- 

- 

14 

275 

87 

148 

76 

159 

46 

15 

- 

- 

- 

- 

- 

- 

16 

244 

85 

170 

76 

172 

51 

17 

- 

- 

- 

- 

- 

- 

18 

242 

85 

191 

76 

179 

56 

19 

240 

85 

201 

76 

181 

59 

TABLE  2.14:  DETAIL  COMPONENT  COSTS 

AUTODIN  II  System 

Based  on  designs  listed  in  Column  1 of  Table  2.4 
Unit:  K$/Mo. 


LOCAL  ACCESS 


BACKBONE  SWITCH 


MILEAGE 


HARDWARE 


COST 

COST 

COST 

3027 

548 

254 

2745 

548 

297 

2523 

548 

339 

2460 

548 

382 

2361 

548 

424 

2305 

548 

466 

2293 

548 

509 

2258 

548 

551 

2216 

548 

636 

2187 

548 

678 

2069 

548 

721 

2078 

548 

763 

1989 

548 

1018 

1982 

548 

1102 

TABLE  2.15! 


DETAIL  COMPONENT  COSTS 


RGC  System 

Based  on  designs  listed  in  column  1 of  Table  2.5. 
Unit:  K$/mo. 
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COST 
(K$/Mo.  ) 


LL  = LOCAL  ACCESS  LINE  COST. 

LH  = LOCAL  ACCESS  HARDWARE  COST. 
SB  = SWITCH  BASIC  COST. 

SM  = SWITCH  MODULAR  COST. 

BL  = BACKBONE  LINE  MILEAGE  COST. 
BH  = BACKBONE  MODEM  COST. 
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14 

15  16 

17  18 

COMPONENT  COST 

VS.  NUMBER 

OF 

BACKBONE 

NODES 

(RGC  SYSTEM  BASED  ON  TABLE  2.15) 
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can  be  approximated  very  closely  by  straight  lines,  and  the  re- 
maining one  (the  local  access  mileage  cost  curve)  can  be  approx- 
imated by  a quadratic  curve. 

In  this  study  we  have  considered  the  effect  on  the  optimum 
design  of  the  unit  cost  change  of  the  local  access  mileage  cost, 
the  backbone  mileage  cost  and  the  switch  basic  cost.  The  results 
are  tabulated  in  Tables  2.16-2.21.  In  these  tables,  we  check  the 
lowest  cost  designs  for  each  set  of  unit  costs  in  order  to  vis- 
ualize the  effect  of  unit  cost  change. 

1 . Effect  of  Unit  Component  Cost  Variations  on  the 

Optimum  Number  of  Backbone  Nodes 

From  Tables  2.16  - 2.21,  we  can  observe  that 
changes  in  the  unit  cost  of  either  the  local  access 
mileage  cost  or  the  backbone  switch  basic  cost  have 
a much  more  pronounced  effect  on  the  shift  of  opti- 
mum number  of  backbone  nodes  than  similar  changes 
in  the  unit  backbone  mileage  cost. 

We  may  suspect  that  the  effect  of  the  unit  com- 
ponent cost  change  on  the  optimum  number  of  backbone 
nodes  has  something  to  do  with  the  magnitude  of  the 
component  cost  under  consideration.  However,  from 
Tables  2.14  - 2.15,  we  see  that  the  backbone  switch 
basic  cost  is  smaller  than  the  backbone  line  mileage 
cost,  yet  the  unit  switch  basic  cost  variation  de- 
finitely has  a stronger  effect  on  the  optimum  number 
of  backbone  nodes.  In  fact,  even  though  the  local 
access  line  mileage  cost  is  much  larger  than  the 
switch  basic  cost,  their  unit  cost  changes  have  com- 
parable effect  on  the  optimum  number  of  backbone  nodes. 

Thus  we  must  seek  some  other  explanation. 
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«^I.OCAL  ACCESS  LINE 
^'■''^UNIT  COST  * 
NUMBER 
OF  BACK- 
BONE  NODES 


a — . 83 
b = .83 
c = .83 
d = 5 

a = 1.33 
b = 1.33 
c = 1.33 
d = 8 

/537 

649 

/536 

/639 

550 

645 

552 

/ 640 

555 

/ 640 

564 

644 

582 

651 

597 

665 

631 

670 

655 

715 

687 

748 

703 

763 

98 
75 
772 
757 
753 
/750 

/742 

756 


TABLE  2.16:  EFFECT  OF  LOCAL  ACCESS  MILEAGE  COST  VARIATION 


Model:  AUTODIN  II 

Based  on  designs  listed  in  column  1 of  Table  2.4 
Unit:  K$/mo. 


a,b,c,d  means 

a = Line  cost  per  mile  per  month  for  a 2.4  KB/S  link 

b = Line  cost  per  mile  per  month  for  a 4.8  KB/S  link 

c = Line  cost  per  mile  per  month  for  a 9.6  KB/S  link 

d = Line  cost  per  mile  per  month  for  a 50  KB/S  link 

The  two  minimum  cost  designs 


d 


^v^UNIT  SWITCH 
COST 

NUMBER  ^"vJ$/MO) 
OE  BACK- 
BONE  NODES 

5,300 

10,600 

15,900 

21,200 

4 

777 

■■ 

819 

841 

5 

750 

m 

802 

/828 

6 

7 

740 

720 

772 

757 

803 

/794 

835 
/ 831 

8 

710 

753 

/795 

838 

9 

702 

/750 

798 

846 

10 

- 

- 

- 

- 

11 

/ 684 

/742 

801 

859 

12 

/ 692 

756 

819 

883 

13 

- 

- 

- 

- 

14 

717 

791 

865 

940 

15 

- 

- 

- 

- 

16 

712 

797 

882 

966 

17 

- 

- 

- 

- 

18 

733 

828 

924 

1019 

19 

742 

843 

944 

1043 
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^\BACKBONE  LINE 
^\UNIT  COST 
NUMBER  ^^($/MO) 
OF  BACK- 
BONE  NODES 

0 

2. 

5. 

10 

4 

670 

721 

798 

927 

5 

646 

698 

775 

✓ 884 

6 

631 

687 

772 

913 

7 

616 

672 

757 

898 

8 

617 

671 

753 

✓ 888 

9 

608 

665 

/750 

892 

10 

- 

- 

- 

- 

11 

✓ 590 

✓ 651 

/742 

895 

12 

✓ 600 

✓ 662 

756 

912 

13 

- 

- 

- 

- 

14 

632 

695 

791 

951 

15 

- 

- 

- 

- 

16 

625 

694 

797 

968 

17 

- 

- 

- 

- 

18 

649 

721 

828 

1007 

19 

662 

734 

843 

1024 

TABLE  2.18:  EFFECT  OF  BACKBONE  LINE  MILEAGE  COST  VARIATION 


Model:  AUTODIN  II 

Based  on  designs  listed  in  column  1 of  Table  2.4: 
Unit:  K$/mo. 

✓ The  two  minimum  cost  designs 


V LOCAL  ACCESS 
LINE  UNIT  * 
^VCOST  ($) 

NUMBER  Nsv 

OF  BACK-  ^ 

BONE  NODES  ^ 


a = 

.83 

b * 

.83 

c = 

.83 

d = 

.5 

3014 

✓ 3009 

✓ 2990 

3021 

3019 

3051 

3099 

3153 

3232 

3263 

3298 

3338 

3627 

3724 

3771 
3695 
/3621 
3636 
/ 3609 
3627 
3673 
3718 


780 
610 
462 
446 

✓ 4396 

✓ 4395 
4437 
4471 


TABLE  2.19:  EFFECT  OF  LOCAL  ACCESS  MILEAGE  COST  VARIATION 

Model:  RGC 

Based  on  designs  listed  in  column  1 of  Table  2.5 
Unit:  K$/mo. 

(*) 

a,b,c,d  means 

a = Line  cost  per  mile  per  month  for  a 2.4  KB/S  channel 

b ■ Line  cost  per  mile  per  month  for  a 4.8  KB/S  channel 

c = Line  cost  per  mile  per  month  for  a 9.6  KB/S  channel 

d = Line  cost  per  mile  per  month  for  a 50.  KB/S  channel 

✓ The  two  minimum  cost  designs 
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X.  UNIT  SWITCH 
N.  COST 

NUMBER 
OF  BACK- 

BONE  NODES  X. 

21,200 

42,400 

63,600 

/ 

84,800 

6 

4653 

4780 

4907 

5035 

7 

4461 

4610 

4758 

4906 

8 

4292 

4462 

4631 

✓ 4801 

9 

4266 

4456 

4647 

4838 

10 

4184 

✓4396 

✓ 4608 

✓ 4820 

11 

✓ 4162 

✓ 4395 

✓ 4628 

4861 

12 

✓4183 

4437 

4691 

4946 

13 

4195 

4471 

4746 

5022 

14 

- 

- 

- 

- 

15 

4207 

4525 

4843 

5161 

16 

4200 

4539 

4878 

5217 

17 

4144 

4504 

4865 

5225 

18 

416! 

4551 

5314 

24 

4278 

4787 

5295 

5805 

26 

4329 

4880 

5431 

5982 

TABLE  2.20:  EFFECT  OF  SWITCH  BASIC  COST  VARIATION 

Model : RGC 

Based  on  designs  listed  i<v  column  1 of  Table  2.5 
Unit:  K$/mo. 


.•lie  two  minimum  cost  designs 


24 

26 


0 

30 

75 

150 

3973 

4296 

mm 

5588 

3739 

4087 

■SB 

5480 

3566 

3924 

m&m 

5358 

3561 

3919 

5352 

/ 3500 

/3858 

✓ 4396 

/ 5292 

✓ 3495 

✓ 3855 

✓ 4395 

✓ 5295 

3526 

3877 

4437 

5348 

3538 

3911 

4471 

5404 

3588 

3963 

4525 

5462 

3603 

3978 

4539 

5474 

3531 

3920 

4504 

5478 

3585 

3971 

4551 

5517 

3753 

4166 

4787 

5822 

3829 

4250 

4880 

5931 

TABLE  2.21:  EFFECT  OF  BACKBONE  LINE  MILEAGE  COST  VARIATION 


Model:  RGC 

Based  on  designs  listed  In  column  1 of  Table  2.5 
Unit:  K$/mo. 


✓ The  two  minimum  cost  designs 
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Now,  if  we  consider  the  component  cost  variations 
relative  to  the  number  of  backbone  nodes  near  the  op- 
timum point,  then  we  observe  a very  close  correlation. 

In  both  Figures  2.27  and  2.28,  at  the  optimum  point  (N  « 11), 
the  slope  of  the  local  access  mileage  cost  (LL)  is  the 
largest,  the  slope  of  the  backbone  switch  basic  cost 
(SB)  is  smaller  but  still  of  the  same  order  of  magnitude, 
and  the  slope  of  the  backbone  line  mileage  cost  is  always 
much  smaller  than  the  other  two.  This  intuitively 
explains  why  the  unit  cost  variation  of  the  local 
access  mileage  cost  and  the  switch  basic  cost  have 
much  more  visible  impact  on  the  optimum  number  of 
backbone  nodes  than  the  unit  backbone  mileage  cost 
variation.  We  thus  conclude  that  the  effect  of  a 
unit  component  cost  variation  on  the  optimum  num- 
ber of  backbone  nodes  is  roughly  proportional  to 
the  rate  of  cost  change  of  that  component  with  re- 
spect to  the  number  of  backbone  nodes  at  the  optimum 
point.  A more  rigorous  justification  of  this  pro- 
perty using  quantitative  methods  is  provided  in 
Appendix  A. 

2.  Effect  of  unit  component  Cost  Variations  on  the 

Optimum  Design  Cost 

. 

We  now  consider  the  effect  of  unit  component 
cost  variations  on  the  total  cost  of  the  optimum  de- 
signs produced.  From  Tables  2.16  - 2.21,  we  can  see 
that  the  unit  local  access  mileage  cost  variation  pro- 
duces the  most  drastic  change  in  the  optimum  design 
cost,  the  unit  backbone  line  mileage  cost  variation 
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produces  less  impact,  and  the  unit  switch  basic  cost 
variation  produces  the  least  amount  of  change.  This 
can  be  explained  intuitively  by  the  fact  that  for  the 
optimum  design,  among  these  three  cost  components,  the 
local  access  mileage  cost  is  always  the  largest,  the 
backbone  line  mileage  cost  is  the  second,  and  the 
switch  basic  cost  is  the  smallest.  (See  Tables  2.14 
and  2.15,  also  Figures  2.27  and  2.28.)  We  can  thus 
conclude  that  the  effect  of  a unit  component  cost 
variation  on  the  optimum  design  cost  is  approxi- 
mately proportional  to  the  cost  of  that  component 
for  the  optimum  design.  A more  rigorous  justifi- 
cation of  this  property  using  quantitative  methods 
is  also  provided  in  Appendix  A. 

F.  Backbone  Cost  Estimates  Using  the  Direct 
Distance  Approximation 


> 


As  mentioned  in  Section  2.3,  the  accurate  design  of  the  back- 
bone network  at  each  step  of  the  switch . selection  algorithm  is 
computationally  too  cumbersome.  Consequently,  an  approximate  but 
computationally  very  fast  technique,  namely  the  Direct  Distance 
Approximation  (DDA) , has  been  developed.  In  this  section  we  verify 
the  adequacy  of  the  DDA  approach  by  comparing  approximate  costs 
versus  actual  design  cost  for  topologies  based  on  the  AUTODIN  II 
requirements. 

First,  three  different  switch  location  strategies  with  5,  11, 
and  19  nodes  respectively  are  selected.  The  locations  are  selected 
using  the  criteria  of  uniform  geographical  distribution  of  the 
switches  over  the  continental  United  States  and  reasonable  match 
between  switch  location  and  AUTODIN  II  installations. 
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Next,  accurate  backbone  designs  are  performed  for  each  of  the 
above  switch  strategies,  with  backbone  traffic  requirements  derived 
from  the  AUTODIN  II  under  the  assumption  that  user  sites 

are  connected  to  the  nearest  backbone  switch.  The  following  assump- 
tions are  used  in  the  design: 


Line  speed: 


50  Kbps 


Line  mileage  cost: 


$5/mile/mo  (Telpak) 


Line  termination  cost:  $425/mo  (Telpak) 


Average  packet  length:  500  bits 


Delay  constraint: 


T < 100  msec 


Protocol  overhead: 


Routing  overhead: 


Routing  algorithm: 


Optimal  multipath 


From  each  of  the  low  cost  designs  thus  generated,  the  coeffi- 
cients for  DDA  are;  namely  RUTPN  (Routing  Penalty) , UTIL  (Average 
Link  Uitlization)  and  FXCPN  (Fixed  Cost  Per  Node,  corresponding 
to  line  termination  charges)  were  obtained  using  the  procedure 
discussed  in  Section  2.3.  The  values  of  the  coefficients  as  a func- 
tion of  number  of  nodes  are  shown  in  Figure  2.29. 

Notice  from  Figure  2.29  the  small  relative  variation  of  the 

DDA  coefficients  over  a wide  range  of  NN.  This  allows  the  use 
of  interpolation  values  for  strategies  with  intermediate  numbers 
of  nodes,  without  having  to  generate  additional  data  points  from 
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initial  network  designs.  The  behavior  of  the  coefficients  versus 
NN  is  as  expected.  The  fixed  cost  per  node  FXCPN  decreases  with 
NN  since  nodal  degree  (i.e.,  number  of  lines  per  node)  decreases 


when  network  size  is  increased  with  total  throughput  kept  fixed. 

Line  utilization  UTIL  decreases  with  NN  since  lines  tend  to  be 
less  efficiently  utilized  in  large  networks  than  in  small  networks. 
Routing  penalty  RUTPN  increases  with  NN,  since  network  connectivity 
decreases  with  NN  and  therefore  the  ratio  between  direct  distance 
and  shortest  path  distance  also  decreases. 

To  verify  the  accuracy  of  the  approximation  we  consider  the 
5-node  topplogy  (see  Figure  2.30(a))  used  to  generate  the  DDA  co- 
efficients (see  Figure  2.29)  and  two  5-node  topologies  obtained  by 
relocating  some  of  the  nodes  and  reoptimizing  the  line  layout  (see 
Figure?  2.30(b)  and  2.30(c)).  Notice  that  point-to-point  traffic  re- 
quirements may  change  after  each  nodal  relocation;  thus  the  re- 
quirement matrix  need  also  be  updated. 

Table  2.22  compares  actual  costs  versus  DDA  approximation  for 
the  above  mentioned  topologies.  For  topology (a)  actual  cost  equals 
DDA  estimate  since  the  DDA  coefficients  were  evaluated  in  corre- 
spondence to  such  topology.  For  the  other  two  topologies  the  error 
is  less  than  10%.  Recalling  that  errors  on  the  order  of  10%  are 
typical  for  heuristic  and  suboptimal  design  procedures  generally 
employed  for  distributed  networks  [GERLA,  1975] , we  conclude  that 
the  DDA  technique  provides  sufficiently  accurate  backbone  cost 
estimates  for  the  BNSP. 

G.  Procedure  Run  Time 

Both  the  cluster  approach  and  the  modularized  add  approach 
have  been  implemented  in  FORTRAN  and  run  through  ARPANET  on  a USC 
PDP-10.  The.  run  times  for  the  various  solutions  obtained  are 
listed  in  Tables  2.23  - 2.25.  From  these  tables,  we  can  observe 
the  following: 
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ACTUAL  DESIGN  COST  DDA  ESTIMATE 


. 

TOPOLOGY 

(a) 

$158  K/MO 

$158 

K/MO 

TOPOLOGY 

(b) 

$171  K/MO 

$168 

K/MO 

TOPOLOGY 

(c) 

$163  K/MO 

$151 

K/MO 

I 


NUMBER  OF 
BACKBONE 
NODES 

ADD  APPROACH 

CLUSTER  APPROACH 

TIME 

(*) 
MCV  ' 

CPU  TIME 

6 

2' 11" 

2 

7 

2' 13" 

2 

3 ' 1 8 " 

8 

2' 35" 

5/4 

- 

9 

2 '31" 

5/4 

3 '26" 

10 

3' 11" 

1 

- 

11 

3' 07" 

1 

3 '20" 

12 

3' 05" 

1 

3 '18" 

13 

5' 19" 

1/2 

3 '16" 

14 

- 

- 

3 '17" 

15 

5' 32" 

1/2 

- 

16 

5 ' 2 9 " 

1/2 

3 '18" 

17 

9 '57" 

1/4 

- 

18 

9*45" 

1/4 

3 '15" 

19 

- 

I 

i 

- 

24) 

i _ 

i 

3 '13" 

TABLE  2.24;  CPU  TIME  VS.  NUMBER  OF  BACKBONE  NODES 


RGC  System  with  AUTOVON  locations  as  candidates 
Computer  used:  PDP-10 


(*)  MC,  the  module  capacity,  is  relative  to  the 
switch  node  capacity  given  in  Table  2.3 
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NUMBER  OF 
BACKBONE 
NODES 

ADD  APPROACH 

CLUSTER  APPROACH 

TIME 

(*) 
MC ' 1 

CPU  TIME 

8 

5*  30" 

5/4 

3*20" 

9 

- 

- 

3 ' 2 4 " 

10 

6 ' 0 3 " 

1 

3' 16" 

11 

- 

- 

3*20" 

12 

5 ' 4 7 " 

1 

3' 18" 

14 

5' 57" 

1 

- 

16 

- 

- 

3' 13" 

17 

9' 45" 

1/2 

- 

20 

- 

- 

3' 11" 

21 

10*05" 

1/2 

TABLE  2.25:  CPU  TIME  VS.  NUMBER  OF  BACKBONE  NODES 

RGC  System 

Add  Approach:  100  best  terminals  as  candidates 

Cluster  Approach:  All  terminals  as  candidates 

Computer  used:  PDP-10 

(*)  MC,  the  module  capcity,  is  relative  to  the 
switch  node  capacity  given  in  Table  2.3 
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1.  On  a given  system  (AUTODIN  II  or  RGC) , the  run  time 
of  the  cluster  approach  is  nearly  constant. 

2.  On  a given  system  (AUTODIN  II  or  RGC),  the  run  time 
of  the  add  approach  increases  as  the  module  size  decreases. 
Moreover,  when  the  module  size  is  large,  the  effect 

of  the  module  size  on  the  run  time  is  only  marginal, 
but  when  the  module  size  becomes  smaller,  the  run 
time  tends  to  be  inversely  proportional  to  the  module 
size.  This  can  be  explained  by  the  fact  that: 

a.  From  Formula  (2.21),  the  run  time  is 
roughly  proportional  to  the  total  number 
of  modules  used. 

b.  If  the  module  size  is  large,  then  the 
capacity  constraint  is  too  loose,  and  hence 
the  module  size  does  not  have  much  effect 
on  the  number  of  modules  used. 

c.  If  the  module  size  is  small,  then  the 
module  capacity  constraint  is  severe,  and 
hence  the  number  of  modules  used  would  be 
roughly  inversely  proportional  to  the  module 
size. 

3.  On  a given  system  (AUTODIN  II  or  RGC),  the  run  time 
of  the  cluster  approach  is  not  affected  by  the  backbone 
candidate  set  size,  while  the  run  time  of  the  add 
approach  is  roughly  proportional  to  the.  candidate 
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set  size  (compare  Table  2.24  with  Table  2.25),  which 
is  in  agreement  with  Formula  (2.21).  This  fact  sug- 
gests the  following  criteria  in  selecting  the  ap- 
propriate procedure  for  the  BNSP: 


a.  If  the  candidate  set  is  small,  then 
use  the  modularized  add  approach  because 
it  gives  better  results  and  does  not  take 
too  much  time. 


b.  If  the  candidate  set  is  large,  then 
we  can  avoid  the  excessive  run  time  by 
either  using  the  cluster  approach,  or  by 
using  the  add  approach  but  only  working 
with  a small  subset  of  the  candidate  set. 
(In  our  experiment  with  the  RGC  system, 
we  have  run  the  add  approach  with  the  100 
best  terminals  as  candidates,  the  resulting 
selections  are  in  fact  better  than  the  se- 
lections made  by  the  cluster  approach  with 
all  the  terminals  as  candidates.  However, 
in  this  case,  the  add  approach  does  take 
much  more  time.) 


4.  In  moving  from  the  AUTODIN  II  system  to  the  RGC 
system,  we  find  that  the  run  time  for  the  add  approach 
is  nearly  doubled  (roughly  3/2  to  2 times) , whereas 
the  run  time  for  the  cluster  approach  is  almost  tri- 
pled. The  AUTODIN  II  system  has  300  user  sites  and 
the  RGC  system  has  450  user  sites.  Thus  in  terms 
of  the  number  of  terminals  in  the  system,  the  add 
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approach  is  nearly  linear,  whereas  the  cluster  ap- 
proach is  definitely  non-linear.  Hence  in  the  case 
of  very  large  systems  with  a large  number  of  ter- 
minals but  a moderate  number  of  backbone  candidates, 
the  add  approach  would  give  better  run  time  than  the 
cluster  approach. 

E | 

5.  For  both  the  RGC  system  and  the  AUTODIN  II  system, 
the  run  time  for  the  cluster  approach  is  comparable 
or  better  than  the  run  time  for  the  add  approach. 

Hence  we  would  expect  for  moderate  size  systems 
the  cluster  approach  would  give  better  or  compar- 
able run  time  than  the  add  approach. 


6.  From  Sections  2.4,  2.5  and  2.6,  the  computational  com- 
plexities for  the  cluster,  add  and  drop  approaches  are 
respectively  as  follows  (assuming  without  using  sophisticated 
nearest  neighbor  search  methods) : 


and 


tcluster 

fcadd 


t 


drop 


0 (Nt2  + Nt  log  Nt)  , 

0 [NTxNBCx(NMS+flog  NT1 ) ] , 

°<Nt*Nbc2), 


where 

Nt  * Number  of  user  locations. 

■ Number  of  backbone  candidates. 

Njjg  * Number  of  backbone  modules  selected  (for  the 
modularized  add  approach) . 
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We  can  observe  the  following: 

a.  Comparing  the  complexity  of  the  cluster 
approach  with  that  of  the  add  approach,  we  find 
that  they  are  in  good  agreement  with  Observations 
1-5  above. 

b.  The  complexity  for  these  three  approaches 
is  in  fact  roughly  comparable. 

Note  that  the  time  complexity  for  all  the  three  approaches 
can  be  improved  (usually  at  the  expense  of  some  additional  storage) 
by  reducing  the  time  required  to  search  for  the  nearest  neighbors 
(user  locations  or  backbone  candidates),  for  example,  by  using  a 
grid  approximation  technique  as  discussed  in  Section  2.4. 
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2 . 8 CONCLUSIONS  AND  FURTHER  RESEARCH 


The  general  problem  of  large,  distributed  network  design  was 
presented.  It  was  recognized  that  the  novel  difficulty  facing  the 
designer  is  the  selection  of  backbone  switch  locations.  Once  such 
locations  are  determined,  the  design  can  be  completed  using  known 
techniques. 

The  problem  solved  here  is  a simplified  version  of  the  most 
general  case,  in  that  it  assumes  that  user  sites  are  connected  with 
point-to-point  connections  to  the  nearest  backbone  node.  It  was 
shown,  however,  that  more  general  problems  can  be  reduced  to  this 
simplified  form  using  preclustering  techniques. 

Three  backbone  switch  location  algorithms  were  proposed,  namely, 
ADD,  CLUSTER,  and  DROP.  The  algorithms  are  of  heuristic  nature. 

Their  computational  complexity  depends  on  the  number  of  user  sites 
Nt  and  number  of  candidate  backbone  locations  NB(,.  In  particular, 
the  complexity  is  0(NT2)  for  CLUSTER,  0(NT  x log  NT  x NB(J  for  ADD, 
and  0(Nt  x Nfi2)  for  DROP. 

The  AD.D  and  CLUSTER  Algorithms  were  programmed  in  FORTRAN  and 
applied  to  two  large  network  examples,  with  1,000  and  10,000  nodes 
respectively.  The  results  show  that  the  ADD  Algorithm  performs  con- 
sistently better  than  the  CLUSTER  Algorithm.  This  can  be  attributed 
to  the  fact  that  ADD  takes  into  account  the  cost  tradeoff  between 
backbone  lines  and  local  access  lines,  while  CLUSTER  does  not.  How- 
ever, ADD  is  generally  more  time  consuming  than  CLUSTER,  especially 
for  large  number  of  backbone  candidates  Nfic,.  The  following  guide- 
lines apply  in  selecting  the  most  effective  algorithm:  if  the  number 
of  candidates  is  much  smaller  than  the  number  of  user  sites,  the 
ADD  Algorithm  is  the  best  choice;  if,  on  the  other  hand,  the  number 
of  candidate  sites  is  very  large  (e.g.,  the  candidate  site  set  con- 
sists of  all  user  sites),  the  CLUSTER  Algorithm  should  be  used. 

Parametric  studies  were  carried  out  to  determine  the  sensitivity 
of  the  solution  to  changes  in  some  of  the  network  parameters.  It  was 
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found  that  the  number  of  backbone  switches  increases  when  the  unit 
local  line  mileage  cost  increases,  and  decreases  when  the  unit  back- 
bone mileage  or  switch  cost  increase.  Furthermore,  the  results 
show  that  the  optimal  number  of  switches  is  more  sensitive  to  the 
variations  in  the  unit  local  access  line  mileage  cost  and  the  switch 
cost,  and  less  sensitive  to  variations  in  the  unit  backbone  line 
cost.  This  is  attributed  to  the  fact  that  the  rate  of  change  of  the 
local  access  and  the  switch  cost  as  a function  of  number  of  backbone 
nodes  is  higher  than  the  rate  of  change  of  the  backbone  line  cost. 

The  results  also  show  that  the  cost  of  the  optimal  design  is 
most  sensitive  to  the  variations  in  the  unit  local  access  line  mileage 
cost,  less  sensitive  to  the  variations  in  the  unit  backbone  line 
cost,  and  least  sensitive  to  the  variations  in  the  unit  switch  cost. 
This  can  be  attributed  to  the  fact  that,  for  most  of  the  designs 
under  consideration,  the  local  access  line  cost  is  larger  than  the 
backbone  line  cost,  which  is  in  turn  larger  than  the  switch  cost. 

Two  different  sets  of  candidate  locations  were  examined  to  de- 
termine the  impact  of  restrictions  on  the  allowable  switch  installa- 
tions. In  one  case  the  candidate  set  consisted  of  approximately  60 
AUTOVON  locations  fairly  well  distributed  geographically.  In  the 
second  case  the  candidates  were  the  100  user  sites  with  highest 
traffic  volume.  The  experimental  results  show  that  network  cost 
is  lower  by  as  much  as  10%  when  user  sites  rather  than  AUTOVON 
sites  are  selected. 

Some  useful  by-products  of  this  study,  beside  the  general  solu- 
tion scheme  for  the  large  distributed  network  design  problem,  are 
as  follows: 


The  direct  distance  approximation  method  (DDA) , 
which  is  a simple,  yet  effective  method  to  es- 
timate the  backbone  network  ccst. 

The  modularized  add  procedure,  which  is  a gen- 
eralization of  the  standard  add  procedure. 
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• The  cluster  procedure,  which  is  a modification 
of  the  COM  procedure  used  in  the  concentra- 
tor location  problem. 

Both  the  modularized  add  procedure  and  the  cluster  procedure  can 
be  applied  to  the  general  warehouse  location  problem. 

The  work  presented  here  lends  itself  to  extensions  in  several 
directions.  Possible  areas  for  future  investigation  are  the  following: 

y 

. • Development  of  lower  bounds  to  assess  the  accuracy 
of  the  heuristics. 

• Refinements  of  the  proposed  algorithms. 

• Extension  of  the  algorithms  to  handle  more  general 
switch  models  (i.e.,  nonlinear  switch  cost;  cost 
dependent  on  switch  location,  etc.) 

• Extension  of  the  algorithms  to  handle  more  general 
local  access  structures  (i.e.,  loops,  distributed 
packet  networks,  etc.) 

• Extension  of  the  algorithms  to  handle  more  general 
backbone  structures  (i.e.,  mixed  terrestrial  and 
satellite  packet  networks,  circuit  switched  networks, 
etc. ) 

• Inclusion  in  the  design  of  other  important  criteria 
such  as  vulnerability,  controllability,  growth 
flexibility,  in  addition  to  line  and  switch  cost 
criteria. 
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APPENDIX  A 


A SIMPLE  ANALYTIC  MODEL  FOR  NETWORK  COST  STUDY 


In  this  appendix  we  present  a very  simple  cost  model  for  the 
network  components  in  terms  of  the  number  of  backbone  nodes.  With 
this  model  we  derive  some  results  relating  the  optimum  design  vari- 
ations and  the  component  cost  variations. 

From  Figures  2.15  and  2.16  we  observe  that,  in  relation  to  the 
number  of  backbone  nodes,  N,  all  of  the  cost  components  can  be 
approximated  by  quadratic  polynomials  in  N.  In  fact,  in  the  range 
of  interest  to  us  (5-20  nodes) , all  except  the  local  access  mileage 
cost  can  be  approximated  very  closely  by  straight  lines.  Thus,  let 


(=local  access  hardware  cost  + backbone  line  hardware  cost 
+ backbone  switch  fixed  cost) , 


be  the  total  network  cost  for  a near-optimum  network  of  N nodes 
We  thus  have 
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and 


C(N)  * LL(N)  + BL(N)  + SB(N)  + H(N), 


CQ  = LL0  + BLq  + SBq  + Hq, 


C1  = LLX  + BLX  + SBX  + Hir 


c2  * ll2. 


(A. 3) 


Taking  the  first  derivative  of  C with  respect  to  N,  we  obtain 


2C 

2n 


C1  + 2C2N. 


The  optimum  occurs  at  = 0.  (With  the  quadratic  approximation 
there  can  be  only  one  minimum.)  Thus,  let 

N = optimum  number  of  backbone  nodes.  1 

f M 

We  have 

~C±  -(LLj+BLj+SBj+i^) 

N " " 2ll^  (A. 4) 

Suppose  the  unit  local  access  mileage  cost  is  changed  by  a pro 
portion  of  p.  Then  LL (N)  is  approximately  changed  to 

LL’(N)  « (1+p)  LL (N) 

* (1+p) LLq  + (1+p)  LLjN  + (1+p)  LL2N2. 

We  consider  the  effect  on  N. 
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Let 


N'tt  = the  new  optimum  resulting  from  the  unit  cost  of  LL 
LL,p 

changed  by  a proportion  of  p, 


and 


ANtt  „ = N' 

LL $ p LL  f p 


N. 


Then  by  Formula  (A. 4) , 


- [ (1+p) LL1+BL1+SB1+H1] 
2 (l+p)LL2 


- (C^+pLL1> 

2 (l+p) LL2"  * 

Hence 

-(Cj+pLLj^)  -C^ 

ANLL,p  = i(l+p)LL2  " 2C^ 


p (c2ll^—c^ll2 ) 
2(l+p)LL2C2 


-p 


LLI+2LL2N 
2 (l+p)LL2* 


(by  (A. 4)) 


However,  notice  that  from  Formula  (A.l), 
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We  thus  have 


By  similar  derivation,  we  can  obtain 


If  p-*-0,  then  2pLL,-*-0.  Hence  it  follows  from  (A. 6)  that 


Equations  (A. 7)  state  that  the  effect  of  a unit  component  cost 
variation  on  the  optimum  number  of  backbone  nodes  is  proportional  to 
the  rate  of  cost  change  of  that  component  with  respect  to  the  number 
of  backbone  nodes  at  the  optimum  point.  (As  observed  in  Section 
2.7.2.E) . 

We  now  consider  the  effect  of  unit  component  cost  variations  or 
the  optimum  design  cost.  Similar  to  the  previous  development, 
suppose  LL(N)  is  changed  to  LL' (N)  with 
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LL'(N)  - (1+p)  LL (N) . 

Let 

C » the  old  optimum  design  cost, 

C'll,p  " the  new  optimum  design  cost  resulted  from  the  unit 
cost  of  LL  changed  by  a proportion  of  p. 

and 

iC.  _ a C'  — P 

LL,p  u LL,p  C* 

Then 

C LL,p  * LL'(N')  + BL(N')  + SB(N')  + H(N') 

“ (1+p)  LL(N’)  + BL(N')  + SB(N')  + H(N'). 

First  we  evaluate  LL(N'). 

LL(N)  = LLq  + LL^  N'  + LL2N'2 

* LLq  + LLX  (N+ANLL  p)  + ll2  (N+ANll  p)2 

« (llq+ll1n+ll2n2)  + anll  p (ll1+2ll2n) 

* LL(N)  + ANjj.  p ^ (N).  (by  (A.  5)) 

By  similar  derivation,  we  can  show  that 
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H(N')«  H (N)  + ANll  -||  (N). 


Substituting  these  values  into  C'^  , we  obtain 


c'li,p  “ (1+p>  [ll(n)  + “ll,p 


au.,ir. 


♦ (BLW+iN^p  *f§»)l  + (SBtNJ-fSN^p  ] 

+ [HOfl+AN^p  |S(N)] 

[LL (N) +BL (N) +SB (N)  +H (N) ] + p(LL(N)  + ANLL  ^fg(N) J 

+ “Ll,p  + * *fi<5>  + 

C + P [LL (N)  + iHj^  p ^(N)l  + iN^  p |§(M>. 


However,  not5.ce  that  since  N is  the  optimum  for  C, 


f§  <N)  * 0. 


We  thus  have 


5'LL,p  “ C ♦ P 'LL(H)«BttfP 


SLL 


It  follows  that 


A^LL,p  " c'll,p  " C 
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" p [LLWHM^p  2i2j(u>l 


* p illw'+HUv)vl,  <»y  (a-6)) 


By  similar  derivations,  we  can  evaluate  ACOT  _ and  ACen  . We  thus 

DLfP  0x5  r p 

have 


r 3LL  . 2 , 


and 


4CU.,p»  P lLL(N,+  2(l+p> LL; 

4eBL,p  * P jSj  (iW(ir»>2>' 


4CSB,p“  p ISB(N)+  2!%  <t§»»2]. 


(A. 9) 


If  p-*-0,  then  YTT+pYLL^  jjjj(N))  etc.  Hence  from  (A. 9),  we 


can  conclude  that 


3C 

3LL 


w LL(N)  , 


3C 

3BL 


* BL(N) , 


(A. 10) 


and 


3C 

3SV 


« SV(N) 


Equations  (A. 10)  state  that  the  effect  of  a unit  component  cost 
variation  on  the  optimum  design  cost  is  proportional  to  the  cost  of 
that  component  at  the  optimum  point. 


i 
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3.  IMPACT  OF  SATELLITE  TECHNOLOGY  ON  THE  DESIGN  OF  LARGE  NETWORKS 
3.1  INTRODUCTION 

The  proliferation  of  data  communications  usages  in  recent  years 
has  stimulated  the  development  and  planning  of  many  nationwide  large 
scale  data  and  computer  communications  networks,  with  total  annual 
communications  costs  for  each  network  in  the  order  of  millions  of 
dollars.  In  this  scenario,  domestic  satellite  communications  play 
an  important  role  in  reducing  network  communications  cost  and  in- 
creasing network  growth  flexibility. 

Satellite  communications  can  be  used  in  four  different  modes  in 
a packet-switching  environment: 

1.  Point-to-point  connections,  replacing  more  costly 
terrestrial  lines. 

2.  Multiple  access  to  a single  channel  from  earth 
stations  located  at  selected  backbone  node  sites  to 
upgrade  the  terrestrial  backbone  network. 

3.  Multiple  access  to  a single  channel  fiom  earth 
stations  located  at  each  backbone  node  site,  replacing 
the  terrestrial  backbone  network  entirely. 

4.  Multiple  access  to  a single  channel  with  one 
earth  station  at  each  terminal  and  computer  site, 

(this  approach  basically  eliminates  all  terrestrial 
lines) . 

The  point-to-point  connection  alternative  has  been  investigated 
before  [GERLA,  1974].  In  this  mode,  the  design  decision  of  choosing 
between  a satellite  link  and  a terrestrial  line  is  not  different 


3.1 
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from  that  of  choosing  a line  from  any  two  different  line  tariffs  as 
far  as  the  cost  is  concerned.  The  other  three  alternatives  require 
a more  complex  design  procedure.  The  purpose  of  this  chapter  is  to 
provide  guidelines  for  the  selection  among  such  alternatives. 

In  comparing  costs  between  a terrestrial  network  and  a satel- 
lite channel,  it  is  not  realistic  to  compare  them  on  a same  re- 
sponse time  basis.  Therefore,  response  times  for  different  config- 
urations are  not  compared.  However,  no  design  proposed  in  this 
study  exceeds  an  end-to-end  packet  delay  of  1/2  second. 


' 
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3.2  NETWORK  MODELS  FOR  SATELLITE  COMMUNICATIONS 


A satellite  communication  channel  of  a certain  bandwidth  or 
capacity  can  be  configured  either  as  a cable  in  the  sky  that  con- 
nects two  earth  stations  (point-to-point  access)  or  as  a facility 
that  is  contended  and  shared  by  several  earth  stations  (multiple 
access).  In  the  point-to-point  mode,  the  satellite  channel  is  used 
just  like  a terrestrial  channel  and  can  be  modeled  in  the  same 
manner,  i.e.,  a line  connecting  two  end  points. 

In  the  multiple  access  mode,  the  allocation  of  the  channel  can 
be  either  static  or  dynamic.  With  the  static  allocation,  the  channel 
is  further  divided  into  subchannels  and  each'  user  is  dedicated  a 
subchannel.  All  subchannels  do  not  have  to  be  assigned  the  same 
capacity.  This  can  be  achieved  by  Time  Division  Multiplexing, 
Frequency  Division  Multiplexing,  or  other  appropriate  multiple 
access  schemes.  Once  a subchannel  is  assigned  to  a user  or  an  earth 
station,  other  users  or  stations  cannot  access  this  subchannel  re- 
gardless whether  it  is  idle  or  not.  However,  any  user  or  earth  sta- 
tion may  receive  information  from  other  subchannels  because  of  the 
broadcast  nature  of  the  satellite  downlink.  This  fixed  allocation 
broadcasting  approach  is  termed  as  Channel  Division  Multiple  Access. 
In  essence,  every  earth  station  is  given  a fixed  allocation  of  the 
uplink  channel,  while  the  downlink  channel  is  broadcast. 

With  the  dynamic  allocation,  the  users  seek  access  to  the  same 
channel  independently.  A set  of  built-in  rules  regulate  the  traffic 
to  recover  information  lost  due  to  interference  of  more  than  one 
user  accessing  the  channel  simultaneously,  and  to  permit  orderly 
access  to  earth  stations  having  traffic  to  send  for  a time  interval 
which  may  or  may  not  be  constant.  Such  multiple  access  schemes  are 
often  referred  to  as  dynamic  access  techniques  or  random  access 
techniques.  The  dynamic  access  schemes  can  be  further  classified 
into  slotted  and  unslotted. 
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In  the  slotted  case,  the  user  gains  access  to  the  satellite 
channel  at  the  beginning  of  a time  slot,  and  maintains  control  of 
the  channel  for  one  or  more  slots,  depending  on  the  reservation 
scheme.  In  the  unslotted  case,  the  user  may  access  the  channel 
at  any  time,  with  no  slot  framing  restrictions. 

In  general,  the  slotted  channel  provides  better  efficiency, 
but  requires  more  elaborate  controls  than  the  unslotted  channel 
[PUENTE,  1975],  [BINDER,  1975],  [ABRAMSON,  1973],  [ROBERTS,  1973], 
[KLEINROCK,  1973],  [CROWTHER,  1973]. 

Multiple  access  satellite  channels  can  be  represented  with  the 
model  shown  in  Figure  3.1,  consisting  of  2N£  + 1 simplex  links  (where 
N£  is  the  number  of  earth  stations)  and  two  hypothetical  store-and- 
forward  switches  . The  uplinks  are  connected  to  one  switch  and  the 
downlinks  are  originated  from  the  other  switch.  A simplex  link 
connects  the  first  switch  to  the  second  one.  The  link  capacity 
values  for  both  the  channel  division  model  and  the  random  access 
model  are  shown  in  Figure  3.1. 

To  compute  average  delay,  optimal  routes  and  throughput  of  a 
mixed  terrestrial  and  satellite  network,  satellite  links  are  replaced 
by  appropriate  models  and  conventional  queueing  formulae  and  routing 
algorithms  are  applied. 


3.4 
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C = Total  channel  capacity 
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3.3  COSTS  FOR  SATELLITE  COMMUNICATIONS 


Many  factors  influence  the  cost  and  capacity  of  a satellite 
channel  used  in  the  multiple  access  mode  with  privately  owned 
earth  stations.  Technology  advancements,  new  applications,  and 
competition  among  domestic  satellite  companies  and  among  earth 
station  component  vendors  are  constantly  changing  the  cost  picture. 
The  cost  figures  used  in  this  section  have  been  derived  from  cost 
estimates  obtained  from  several  different  sources  [CACCIAMANI,  1975], 
[JANKY,  1975],  [ALDRICH,  1975],  [PONTANO,  1975],  [LUSIGNAN,  1972], 
[MITCHELL,  1975],  [DAVIS,  1975],  [WHITWORTH,  1974],  [O'NEIL,  1974]. 
Interpolation  and  extrapolation  techniques  were  used  to  supplement 
the  data  points  available. 

Figure  3.2  shows  the  required  normalized  carrier/noise  ratio 
(C/N) , as  a function  of  error  rate  for  a satellite  channel  with  a 
capacity  of  1.344  MB/s  [PUENTE,  1975].  The  use  of  four  phase 
phase-shif ted-key  coherent  modulation/demodulation  and  3/4  convolu- 
tional encoding  threshold  decoding  techniques  is  assumed. 


Suppose  Pbe  = 10 


-8 


(Pj^e  - prob.  of  bit 
error) 


Then 


C/N  = 13.2  db 


(obtained  from  Figure  3.2) 


Hence  C/Nq  = 13.2  + 10  log^  1.1  x 10 
=73.6  db/Hz 


(C/N  is  the  total 
o 

carrier-noise-ratio 
and  1.1  x lO^Hz  is  the 
bandwidth  required  for 
a 1.344  MB/s  channel) 


Therefore  downlink  C/Nq~  73.6  db/Hz 


(Usually,  uplink  C/Nq 
is  a couple  of  db  higher 
than  the  downlink  C/N 
and  downlink  C/Nq  is 
almost  the  same  as  total 
C/N.) 
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PROBABILITY  OF 
BIT  ERROR  RATE 


1.344  MB/S  REQUIRING 
1.1  MHz  BANDWIDTH 


NORMALIZED  CARRIER-NOISE-RATIO 


FIGURE  3.2:  NORMALIZED  CARRIER-NOISE->RATIO  (C/N) 
VS  PROBABILITY-OF-BIT-ERROR-RATE  (P, 
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Downlink  C/NQ  « [EIRP]safc  - Lp  + [G/T]fiS  - K, 

where,  [EIRPl  * satellite  effective  isotropic  power 

Saw 

IG/Tl^e  = station  gain  versus  temperature  ratio 
(receiving  gain) 

LP  = downlink  path  loss 

= 196.8  db  at  4GHz  and  at  5°  elevation 
angle 

K = Boltzman's  constant 
= -228.6  dbw/Hz  - °K 


r- 


N 

- 


Therefore,  the  minimum  power  requirement  is: 
[EIRP ] gat  + [G/T] ES  =41.8  dbw 
Assuming  6.0  db  as  operating  margin. 


* 

:i 


. [EIRP] gat  + [G/T]es  = 47.8  + 10  log1Q  (CC/1.344)  dbw 
where,  CC  = channel  capacity  in  Mbps. 

The  dominating  factor  in  costing  an  earth  station  is  its  gain- 
temperature  ratio,  [G/T] EE . The  higher  the  value  of  [G/T] Eg , the 
more  expensive  the  earth  station,  the  lower  the  satellite  requirement 
[EIRP]gat,  and  thus  the  lower  the  space  segment  cost.  To  obtain  a 
satellite  channel  of  1.344  MBPS,  the  required  sum  of  [EIRP]gat  and 
[G/T] EE  is  47.8  dbw.  The  specific  values  for  [EIRP]gat  and  [G/T]Eg 
to  achieve  a least  cost  arrangement  depend  on  satellite  carrier 
tariffs,  available  earth  stations  and  number  of  earth  stations 
sharing  the  same  channel. 
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Figure  3.3  gives  the  estimated  monthly  space  segment  cost  (in- 
cluding redundancy  protection)  as  a function  of  [EIRP]  . Figure 
3.4  shows  estimated  earth  station  cost  as  a function  of  [G/T]Fg,  i 
eluding  antenna  with  seat  and  mounting,  transmitter  (TWT) , receiver 
(LNA) , up  converter,  down  converter  and  modem.  In  order  to  trans- 
late earth  station  purchase  price  into  monthly  costs,  the  following 
assumptions  are  used: 


Building  & site  costs 


Monthly  cost  based  on 
10  years  amortization 
10%  annual  interest 
(excluding  O & M) 


1.6875%  of  purchase  price/mo 


Operating  and  mainte 
nance  (O  & M) 


1.0%  of  purchase  price/mo 


Total  monthly  cost 
(with  0 & M) 


2.6875%  purchase  price/mo 


MONTHLY  COSTS 
(K$/MO) 


[EIRP] 


FIGURE  3.3:  SPACE  SEGMENT  COSTS 
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3.4.  COST  IMPACT  OF  PLACING  AN  EARTH  STATION  AT  EVERY  USER  SITE 


It  has  been  suggested  that  satellite  communication  channels 
with  dynamic  access  scheme  can  be  very  cost-effective  in  a data  and 
computer  communications  environment  with  a large  number  of  geograph 
ically  widely  dispersed  terminals  and  computers  with  bursty  traffic 
pattern.  This  contention  is  based  on  the  fact  that  channel  utiliza 
tion  is  much  higher  in  a broadcast  mode  than  in  a point-to-point 
connection  mode,  and  that  earth  stations  will  become  quite  inexpen- 
sive in  the'  near  future.  In  this  section,  we  investigate  the  con- 
ditions under  which  this  contention  is  valid. 


N = Total  number  of  terminals  and  computers  in 
the  data  communication  system  under  design 


fsat(EIRP)  = Monthly  space  segment  cost  for  the  speci 
fied  EIRP. 


f c (G/T)  = Monthly  earth  cost  for  the  specified  G/T 


EIRP  + G/T  = Total  required  sum  of  [EIRP]gat  and  [G/T] Eg 
for  the  channel  under  consideration. 


Then,  the  monthly  satellite  communications  costs  for  the  system  * 

f . (EIRP)  + N f„-<  (G/T).  This  cost  is  minimized  by  proper  choice 
sat  es 

of  values  for  [ElRP]gat  and  [G/T]Eg. 

One  cam  determine  whether  satellite  communications  are  less 
expensive  than  terrestrial  communications  by  comparing  the  minimized 
satellite  communications  costs  with  the  optimized  cost  for  the  same 
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system  using  terrestrial  lines.  There  is  no  easy  way  to  draw 
definite  conclusions  since  too  many  factors  including  terminal 
and  computer  locations,  speed,  message  characteristics,  perfor- 
mance requirements,  tariffs,  access  methods,  etc.,  have  impact 
on  cost.  Instead,  an  example  is  presented  here  in  order  to  ob- 
tain insight  into  the  problem. 

Figure  3.5a  shows  the  topological  representation  of  an  integrated 
data  communications  system  that  contains  35  separate  Defense  ADP 
systems  and  a total  of  87  computers  and  1103  terminals.  The  network 
shown  consists  of  8 nodes  interconnected  by  terrestrail  lines.  The 
computers  and  terminals  generate  a total  of  1.26  MBPS  during  the 
busy  hour.  The  total  line  and  communications  hardware  (including 
modems,  multiplexers,  concentrators,  terminal  control  units,  and 
switches)  is  about  $898K  per  month. 

To  evaluate  the  cost  of  using  satellite  communications  to 
replace  the  system's  terrestrial  lines,  the  following  assumptions 
are  made: 

1.  The  channel  efficiency  for  using  a multiple  access 
scheme  is  about  25%  (the  low  efficiency  is  due  to:  con- 

tention, partially  occupied  slots,  overhead  traffic, 
etc.).  Thus,  the  required  channel  capacity  is  1.26  x 

4 = 5.04  MBPS.  The  required  total  [EIRP]  sat  + [G/T]_e 

S 04 

is  47.8  + 10  log1Q  dbW,  or  53.5  dbW. 


2.  A switch  is  needed  at  each  earth  station  to  handle 
the  traffic,  perform  access  schemes,  control  traffic 
flow,  carry  out  local  switching  functions,  and  buffer 
messages.  The  switch  purchase  cost  is  assumed  to  be 
$50,000.  Monthly  cost  is  obtained  by  applying  the 
coefficient  of  2.6875%  to  the  purchase  cost,  as  done 
previously  for  the  earth  station  cost. 
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NUMBER  ON  LINKS  INDICATES  MULTIPLE  50  KBPS  LINES 

TOTAL  NETWORK  COST  (EXCLUDING  SWITCHES)  = $173K/MO 

FIGURE  3.5a:  BACKBONE  NETWORK  WITHOUT  EARTH  STATIONS 
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NUMBER  ON  LINKS  INDICATES  MULTIPLE  50  KBPS  LINES 
TERRESTRIAL  COMMUNICATIONS  COST  - $7lK/MO 
SATELLITE  COMMUNICATIONS  COST  - $80K/MO 

FIGURE  3.5b:  BACKBONE  NETWORK  WITH  EIGHT  EARTH  STATIONS 
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ALBANY 


SWITCH  WITHOUT  EARTH  STATION 
SWITCH  WITH  EARTH  STATION 


NUMBER  ON  LINKS  INDICATES  MULTIPLE  50  KBPS  LINES 
TERRESTRIAL  COMMUNICATIONS  COST  - $75K/MO 
SATELLITE  COMMUNICATIONS  COST  - $63K/MO 


FIGURE  3.5c:  BACKBONE  NETWORK  WITH  SIX  EARTH  STATIONS 
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O SWITCH  WITHOUT  EARTH  STATION 

• SWITCH  WITH  EARTH  STATION 


NUMBER  ON  LINKS  INDICATES  MULTIPLE  50  KBPS  LINES 
TERRESTRIAL  COMMUNICATIONS  COST  - $94K/MO 
SATELLITE  COMMUNICATIONS  COST  - $54K/MO 
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With  the  above  assumptions,  it  is  found  that  the  least  cost  ar- 
rangement is  to  use  a full  transponder  of  33  dbW  EIRP  at  $180, 000/mo 
(Figure  3.3)  and  earth  stations  of  20.5  dbW  G/T  at  about  $100,000  each 
without  the  switch,  and  at  about  $150,000  with  the  switch.  For 
a total  of  1190  computers  and  terminals,  the  monthly  cost  for  1190 
earth  stations  with  switch  is  0.026875  x 1190  x 150,000  or  $4,797K. 

The  total  satellite  communications  cost  with  the  space  segment  is 
$4,977K  per  month.  This  is  much  higher  than  the  communication  cost 
with  only  terrestrial  lines  which  is  $898K  per  month. 

However,  even  though  there  are  a total  of  1190  computers  and 
terminals,  the  distinct  geographic  locations  are  about  300.  Assum- 
ing that  terminals  and  computers  at  a same  site  can  share  the  same 
earth  station  and  switch,  the  satellite  communications  cost  is  re- 
duced to  l,386K/mo. 

Based  on  the  above  cost  assumptions  we  may  conclude  that  the 
terrestrial  network  implementation  is  more  cost-effective  than  the 
satellite  network  implementation  for  the  present  defense  ADP  require- 
ments with  300  geographically  distinct  user  sites. 

Next,  we  generalize  the  above  results  and  compare  terrestrial  and 
satellite  network  cost  for  different  user  site  numbers,  yet  keeping 
the  total  throughput  requirement  at  1.26  MBPS.  For  the  satellite 
network  cost  estimate  we  assume  that  both  ground  station  and  switch 
cost  increase  when  the  number  of  sites  decreases,  due  to  the  fact 
that  bandwidth  and  I/O  interface  requirements  per  site  increase.  We 
estimate  that  ground  station  purchase  price  varies  from  $100K  to  $205K 
and  switch  purchase  price  varies  from  $50K  (regular  IMP)  to  $500K 
(Pluribus  IMP)  when  the  number  of  sites  decreases  from  300  to  8. 

Furthermore,  we  assume  that  the  cost  of  the  space  segment  de- 
creases when  the  number  of  ground  stations  decreases,  due  to  better 
channel  utilization  and  lower  satellite  EIRP  requirements  (more 
powerful  ground  stations) . The  satellite  segment  monthly  cost  es- 
timate ranges  from  $180K  (for  300  sites)  to  $40K  (for  8 sites) . 
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For  the  terrestrial  network  cost  estimate,  we  interpolate  the 
results  obtained  in  a previous  study  [NAC,  1975]  for  three  network 
designs  with  8,  27  and  300  user  sites  respectively. 

Figure  3.6  shows  the  cost  trends  for  satellite  and  terrestrial 
networks  as  a function  of  number  of  sites.  We  notice  that  terres- 
trial and  satellite  costs  are  approximately  equal  for  a number  of 
sites  in  the  range  between  100  and  150.  Satellite  cost  is  higher 
than  terrestrial  cost  for  large  number  of  sites  and  is  lower  for 
small  number  of  sites. 

In  general,  for  a given  throughput  value,  there  is  a critical 
number  of  user  sites  above  which  the  terrestrial  network  is  more 
cost-effective,  and  below  which  the  satellite  network  is  more  cost- 
effective.  Intuitively,  this  result  is  justified  by  the  fact  that 
earth  station  cost  (which  is  the  predominant  contribution  to  satel- 


lite network  cost)  is  increasing  almost  linearly  with  number  of 
sites,  while  terrestrial  line  cost  (which  is  the  predominant  con- 
tribution to  terrestrial  network  cost)  increases  much  less  than 
linearly  with  the  number  of  sites  served,  particularly  if  a two- 
level  hierarchical  topology  (backbone  and  local  access)  is  used 
where  the  number  of  sites  is  large. 

In  the  previous  example  with  300  user  sites,  the  purchase 
price  of  an  earth  station  with  switching  capability  was  assumed 
to  be  $150K,  and  the  monthly  co^t  2.6875%  of  the  purchase  price. 
Should  the  two  assumptions  vary,  the  conclusion  may  be  different. 
Figure  3.7  shows  the  satellite  communications  cost  as  a function  of 
earth  station  costs  with  monthly  rate  as  a parameter.  It  can  be 
seen  that  with  monthly  cost  at  2.6875%  of  the  purchase  price,  satel- 
lite communications  may  cost  less  than  terrestrial  communications 
if  the  earth  stations  cost  less  than  $90K  each.  For  monthly  rate 
at  5%,  satellite  communications  cost  is  less  expensive  if  the  earth 
stations  cost  less  than  $50K  each.  And  for  monthly  rate  at  1.5%, 
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FIGURE  3 . 6 i COMMUNICATIONS  COST  VERSUS  NUMBER  OF  SITES 
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3.5.  USING  EARTH  STATIONS  AS  TRAFFIC  CONCENTRATION  POINTS 


The  main  conclusion  obtained  from  the  last  section  is  that  the 
placement  of  one  earth  station  at  each  terminal  (or  at  each  user 
site)  in  a large  data  communications  system  is  generally  not  cost-effec- 
tive and  satellite  communications  become  attractive  only  for  inter- 
connection of  distant  points  with  heavy  traffic  requirements.  There- 
fore satellite  communications  may  be  used  to  reduce  overall  communi- 
cations costs  in  the  following  fashion.  First,  we  concentrate  ter- 
minal traffic  by  connecting  terminals  to  a few  selected  points;  then, 
we  interconnect  these  concentration  points  via  satellite.  The  problem 
of  locating  the  concentration  points  is  analogous  to  the  location  of 
backbone  nodes  [NAC,  1975],  [MCGREGOR,  1975],  [CHOU,  1974],  and  the 
same  methodology  can  be  used.  Although  the  technology  is  too  young 
and  the  problem  too  complex  to  draw  definite  conclusions  about  the 
use  of  earth  stations  as  concentration  points  in  a large  network, 
some  examples  are  investigated  to  provide  preliminary  tradeoffs. 

For  the  35  Defense  ADF  systems,  it  was  shown  that  cost  savings 
are  achieved  using  a packet- switched  backbone  network  [NAC,  1976]. 

One  of  the  proposed  topologies  consists  of  8 backbone  nodes,  each 
equipped  with  a Pluribus  IMP.  Another  configuration  considered 
consists  of  27  switches  with  a total  of  over  100  regular  IMP'S 
clustered  at  such  switches.  (Several  IMP'S  may  form  a cluster  or 
Super  IMP  to  provide  higher  throughput  and  higher  I/O  capacity) . 

Since  backbone  nodes  themselves  are  traffic  concentration  points, 
their  locations  are  natural  candidates  for  earth  station  sites. 

In  the  following  discussion,  it  is  assumed  that  no  significant 
additional  cost  to  the  switches  is  needed  to  handle  the  satellite 
communication  protocols.  Thus,  total  switch  cost  is  assumed  equi- 
valent for  satellite  and  terrestrial  approach,  and  is  not  included 
in  the  following  cost  comparisons. 
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The  cost  of  a terrestrial  27  node  backbone  network  (modem, 
line  terminal  charges,  and  mileage  charges)  is  $373K/mo.  Using  the 
satellite  channels,  the  total  cost  including  earth  stations  plus 
space  segment  is  only  $188K/mo.  (Space  segment  monthly  cost  is 
estimated  at  $54K/mo,  and  earth  station  purchase  cost  is  estimated 
at  $185K  each.)  For  the  8 node  backbone  network,  the  terrestrial 
network  cost  is  $173K/mo  and  the  satellite  communications  cost  is 
$85K/mo.  (Space  segment  monthly  cost  is  estimated  at  $40K/mo,  and 
earth  stations  purchase  cost  is  estimated  at  $205K  each.) 

The  above  cost  figures  show  that  satellite  channels  can  provide 
considerable  savings  when  ground  stations  are  located  at  traffic 
concentration  points  in  a very  large  data  communications  system. 

Instead  of  replacing  the  terrestrial  backbone  network  in  its 
entirety  with  satellite  channels,  we  may  consider  a hybrid  satellite- 
terrestrial  implementation  in  which  a terrestrial  backbone  structure 
of  low  capacity  is  preserved  for  low  delay  traffic  (e.g*  acknowledge- 
ments, control  messages,  interactive  traffic)  while  the  satellite 
channel  provides  a high  bandwidth,  low  cost  path  for  bulk  traffic. 

To  obtain  some  preliminary  estimates  on  the  cost-effectiveness  of 
the  hybrid  satellite-terrestrial  approach,  we  study  the  following 
hybrid  implementations  for  the  AUTODIN  II  network: 

1.  Ground  stations  at  all  eight  switch  locations, 

1. e.,  McClellan,  Norton,  Timber,  Gentile,  Albany, 

Andrews,  Ft.  Detrick,  and  Hancock. 

2.  Ground  stations  at  McClellan,  Norton,  Tinker, 

Gentile,  Albany,  and  Andrews. 

3.  Ground  stations  at  McClellan,  Norton,  Tinker, 

Gentile,  and  Albany. 
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In  each  case,  the  topology  of  the  backbone  network  is  unchanged.  The 
capacity  on  any  terrestrial  link  connecting  two  switch  sites  with 
earth  stations  is  reduced  to  50  Kbps.  After  link  capacity  reduc- 
tion, the  terrestrial  network  cannot  handle  the  original  throughput. 
However,  satellite  channel  and  terrestrial  network  combined  satisfy 
the  throughput  requirements. 

For  the  8 earth  station  case,  the  terrestrial  communications 
cost  is  $71K/mo,  the  satellite  communications  cost  is  $80K/mo,  and 
the  total  is  $151K/mo,  a savings  of  $22K/mo  with  respect  to  an  en- 
tirely terrestrial  implementation.  In  this  case,  the  satellite 
channel  handles  0.864  MBPS  of  information  traffic. 

For  the  case  with  6 earth  stations,  terrestrial  cost,  satel- 
lite cost,  and  total  cost  are  $75K/mo,  $65K/mo  and  $138K/mo,  respec- 
tively. The  savings  amount  to  $35K/mo  with  respect  to  the  terres- 
trial implementation.  The  satellite  channel  handles  0.777  MBPS  of 
information  traffic. 

For  the  5 earth  station  case,  the  costs  are  $94K/mo,  $54K/mo, 
and  $148K/mo,  respectively.  The  savings  amount  to  $25K/mo,  and  the 
satellite  channel  handles  0.59  MBPS  of  information  traffic. 

The  6 earth  station  implementation  therefore,  is  the  most  cost- 
effective  solution.  The  results  are  summarized  in  Table  3.1  and  in 
Figure  3.8. 
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BACKBONE  COMMUNICATIONS 
COST  (K$/MO) 


HYBRID  IMPLEMENTATION 


© PURE  SATELLITE  IMPLEMENTATION 


NUMBER  OF  EARTH  STATIONS 


FIGURE  3.8:  BACKBONE  COMMUNICATIONS  COST  (EXCLUDING  SWITCH  COST) 

FOR  A HYBRID  TERRESTRIAL  - SATELLITE  IMPLEMENTATION. 
AS  A FUNCTION  OF  NUMBER  OF  EARTH  STATIONS 
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3.6  CHANNEL  EFFICIENCY  AND  COST 

Among  the  three  basic  multiple  access  broadcast  approaches/ 
unslotted  dynamic  allocation,  slotted  dynamic  allocation,  and  fixed 
allocation  broadcasting,  the  slotted  dynamic  allocation  has  received 
most  attention  in  the  packet- switching  environment.  A variety  of 
schemes  have  been  published.  Every  scheme  has  been  analyzed  and 
evaluated  on  the  basis  of  channel  efficiency,  and  the  evaluation 
was  based  on  the  assumption  that  every  packet  is  a full  length 
packet,  i.e.,  busy  slots  are  fully  utilized.  However,  average 
packet  length  is  in  general  much  smaller  than  maximum  packet  length, 
especially  for  systems  with  large  user  population  and  busy  traffic 
characteristics,  for  which  multiple  access  to  a dynamically  shared 
satellite  channel  is  most  attractive.  In  ARPANET,  for  example,  the 
average  packet  length  is  about  250  bits  [KLEINROCK,  1975]  , much  less 
than  the  full  packet  size  of  1000  bits.  Since  a full  slot  is  about 
1400  bits,  including  synchronization  overhead,  (assuming  a full 
packet  length  of  about  1000  bits) , satellite  channel  efficiency  with 
the  slotted  scheme  would  be  very  poor  if  the  average  packet  length 
is  250  bits  like  in  ARPANET.  Therefore,  unslotted  dynamic  alloca- 
tion and  fixed  allocation  broadcasting  should  not  be  a priori  dis- 
carded as  less  efficient  than  the  slotted  dynamic  allocation  schemes. 

In  particular,  the  conclusion  of  the  last  section  was  that  a satellite 
communication  channel  can  be  shared  more  cost-effectively  by  a smaller 
number  of  earth  stations  or  users,  mostly  with  heavy  traffic  (high 
duty  cycle)  than  by  a larger  number  of  users  with  light  traffic  (low 
duty  cycle) . A fixed  allocation  broadcasting  scheme  was  used. 

The  accurate  evaluation  of  the  channel  efficiency  achieved  with 
different  multiple  access  schemes  should  take  into  account  a variety 
of  channel  control  parameters  and  traffic  characteristics,  and  is 
beyond  the  scope  of  this  study.  Previous  cost  estimates  were  based 
on  a 25%  channel  efficiency.  Figure  3.9  shows,  for  two  cases,  the 
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variations  in  communications  cost  as  the  assumptions  on  the  satel- 
lite channel  efficiency  vary.  One  case  is  based  on  the  configura- 
tion with  earth  stations  placed  at  every  one  of  the  1190  terminals 
and  computers.  The  other  case  is  for  the  hybrid  configuration  with 
8 earth  stations,  and  a "thin"  8 node  backbone  terrestrial  network. 

From  the  results,  it  can  be  seen  that  channel  efficiency  has  a 
rather  modest  impact  on  the  8 station  configuration,  and  a much 
more  pronounced  impact  on  the  configuration  with  1000  plus  stations. 
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3.7  CONCLUSION  AND  FUTURE  RESEARCH 


In  the  environment  of  large  data  and  computer  communications 
systems,  domestic  satellite  communications  will  play  an  important 
role  in  reducing  communications  costs.  In  using  satellite  communica- 
tions as  point-to-point  connections,  the  decision  of  choosing  be- 
tween a satellite  link  and  a terrestrial  line  is  not  different  from 
that  of  choosing  a line  from  any  two  common  carrier  alternatives  as 
far  as  the  cost  is  concerned.  If  the  channel  is  shared  by  many 
stations  in  a multiple  access  mode,  a design  with  a small  number  of 
earth  stations,  mostly  with  heavy  traffic  requirements,  is  more 
likely  to  achieve  better  savings  than  a design  with  a large  number 
of  earth  stations,  mostly  with  light  traffic  requirements.  In  other 
words,  satellite  communications  are  more  cost-effective  in  supplement 
ing  or  replacing  high  capacity  links  in  the  terrestrial  backbone 
rather  than  replacing  the  backbone  network  itself.  Although  the 
cost  estimates  used  in  this  study  are  rather  crude,  the  general 
trends  and  conclusions  should  be  valid  for  a wide  range  of  varia- 
tions of  cost  parameters. 

Further  research  is  required  in  the  following  areas: 

• Interaction  between  response  time  performance,  topological 
configuration  and  cost  for  a hybrid  terrestrial  and  satellite 
network. 

• Efficiency  of  different  multiple  access  schemes  under 
different  realistic  traffic  environments. 

• Routing,  flow  control  and  priority  structures  in  a hybrid 
system  combining  satellite  and  terrestrial  communications. 
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CHANNEL  PRIORITIES  AND  THEIR  IMPACT  ON  DATA  NETWORK  DESIGN 
AND  PERFORMANCE 


4.1  SUMMARY 


The  impact  of  priorities  on  the  performance  of  a packet  net- 
work is  twofold:  (1)  better  utilization  of  network  resources;  and 

(2)  prevention  of  blocking  for  critical  traffic. 

Three  main  types  of  network  priority  are  identified:  (1) 

channel  priority;  (2)  access  priority;  and  (3)  path  priority.  In 
this  study  we  focus  our  attention  on  channel  priorities.  First, 
the  general  expression  of  end-to-end  delay  for  channel  priority 
structures  is  derived,  as  a function  of  the  various  network  pa- 
rameters, and  alternative  priority  strategies  are  presented.  The 
strategies  are  then  evaluated  for  the  proposed  AUTODIN  II  network 
topology  and  traffic  environment. 

Substantial  performance  improvements  with  respect  to  the  non- 
priority  case  were  observed  in  the  AUTODIN  II  application.  The 
major  results  for  the  two  class  priority  strategy  (high  level  = 
critical  + subnet  control  packets;  low  level  = regular  packets) 
are  summarized  below: 


• 50%  reduction  in  end-to-end  delay  for  the 

high  priority  packets. 


• 2%  increase  in  end-to-end  delay  for  the 

low  priority  packets. 


• 20%  reduction  in  average  delay  (total  of 

both  classes). 


• 20%  bandwidth  improvement  (assuming  only 

one  outstanding  packet  between  source  and 
destination) . 
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• 20%  store  and  forward  buffer  savings . 

• Full  protection  of  high  priority  traffic  from 
blocking  in  case  of  network  congestion. 

The  performance  results  for  the  various  priority  cases  are  sum- 
marized in  Table  4.1. 

The  major  conclusion  of  this  study  is  that  a 2-level  channel 
priority  structure  is  the  most  cost-effective  strategy  for  the 
AUTODIN  II  requirements  since  it  provides  all  of  the  benefits  that 
can  be  obtained  from  channel  priorities  with  the  minimum  imple- 
mentation cost.  An  additional  priority  level  (for  short  data  packets, 
or  for  interactive  packets)  is  recommended  only  if  response  time 
requirements  for  interactive  traffic  are  very  tight  or  if  Host-to- 
Host  bandwidth  can  be  significantly  improved  by  reducing  Host  pro- 
tocol message  delay. 


: 
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4.2  INTRODUCTION  AND  BACKGROUND 


The  purpose  of  priorities  in  a packet  switched  network  is 
twofold : 


1.  To  make  efficient  use  of  network  resources 
during  normal  traffic  conditions,  and 

2.  To  guarantee  delivery  of  important  traffic 
during  network  congestion. 

The  first  objective  - better  network  utilization  - is  achieved 
by  giving  higher  priority  to  packets  with  tight  response  time  require- 
ments, to  short  packets  and  to  subnet  control  messages  (e.g.,  end- 
to-end  acknowledgements).  Typically,  the  priority  implementation 
consists  of  placing  priority  packets  ahead  of  non-priority  packets 
on  trunk  queues  and  network  access  queues,  or  of  selecting  low  de- 
lay paths  (e.g.,  terrestrial  path  in  a mixed  terrestrial  and  satellite 
configuration) . 

The  second  objective  - congestion  protection  for  privileged 
traffic  - is  achieved  by  providing  higher  priority  to  such  traffic. 

In  particular,  if  the  network  becomes  congested,  privileged  traffic 
is  still  accepted  into  the  network  while  regular  traffic  may  be 
denied  access  until  the  congestion  is  removed.  In  other  words, 
the  network  is  "non-blocking"  for  privileged  traffic,  while  it  may 
introduce  a blocking  delay  for  regular  traffic. 

We  may  define  the  following  types  of  packet-switching  priorities: 


1 . Store-and-Forward  (or  Channel)  Priorities:  These 

priorities  apply  to  the  management  of  trunk  queues  and 
associated  buffers.  The  priority  scheme  used  is  of  the 
"Head  of  the  Line"  type  (i.e.,‘  higher  priority  packets 
are  transmitted  before  lower  priority  ones,  and  service 
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is  first-come  first-served  within  each  priority  class) . 
Channel  priority  can  be  pre-emptive  or  non-pre-emptive . 
Buffer  priority  relates  to  the  seizing  of  S/P  buffers 
by  incoming  packets.  When  buffer  priority  is  imple- 
mented, a high  priority  packet  arriving  at  a nodal  pro- 
cessor with  no  free  S/F  buffers  will  steal  a buffer  from 
a lower  priority  packet.  End-to-end  protocols  provide 
for  the  retransmission  of  discarded  low  priority  pack- 
ets. 

2.  Path  Priorities;  In  some  routing  implementations 
two  or  more  paths  may  be  used  simultaneously  between  a 
given  node  pair  (e.g.,  minimum  hop  path  and  minimum 
delay  path,  or  terrestrial  path  and  satellite  path,  etc.). 
Depending  on  the  application,  one  path  may  be  more  preferable 
than  another.  For  example,  the  min  hop  path  is  protected 
against  looping,  and  should  be  used  for  packets  for  which 
loop  free  transmission  is  an  essential  requirement.  Al- 
ternate or  multiple  paths,  on  the  other  hand,  typically 
provide  higher  bandwidth  than  the  min  hop  path,  but  may 
introduce  occasional  loops.  Therefore,  such  paths  should 
be  used  for  high  bandwidth  requirements,  for  which  looping 
is  less  critical  than  bandwidth.  In  a mixed  terrestrial 
and  satellite  implementation,  the  terrestrial  path  offers 
lower  bandwidth,  but  also  lower  delay  than  the  satellite 
path,  and  therefore,  may  be  preferred  for  low  bandwidth, 
high  response  time  requirements  (e.g.,  subnet  control 
packets).  In  general,  the  assignment  of  priorities  for 
the  use  of  high  quality  paths  can  greatly  improve  network 
efficiency.  As  an  example,  in  a mixed  terrestrial  and 
satellite  network  configuration  the  following  priorities 
may  be  implemented  on  the  terrestrial  path: 
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Class  1:  Subnet  control  messages , 

Class  2:  Interactive  traffic  and  real  time  traffic 

(e.g.,  speech),  and 

Class  3:  File  transfers. 

3.  Access  Priorites:  Input  traffic  is  divided  into 

access  priority  classes  according  to  its  relative  im- 
portance. Based  on  such  priorities,  and  on  traffic 
load,  a selective  throttling  of  input  rates  is  exer- 
cised by  flow  control  and  stability  control  procedures. 
The  access  priority  implementation  is  closely  related 
to  the  subnet  end-to-end  control  structure.  For  ex- 
ample, in  a "virtual  circuit"  subnet  structure  re- 
quiring reservation  of  network  resources  (e.g.,  con- 
nection table  entry,  buffer  storage,  sequence  number, 
etc.)  prior  to  the  acceptance  of  a message  into  the 
network,  the  access  priority  is  applied  to  the  queue 
of  the  user  processes  competing  for  such  resources. 

In  "datagram"  subnet  structures  which  allow  injection 
of  packets  in  the  net  without  prior  reservation,  a 
separate  stability  control  procedure  is  required  to 
determine  the  lowest  priority  level  that  can  be  cur- 
rently accepted.  Input  packets  with  priority  below 
such  minimum  are  discarded  at  the  entry  switch. 

In  this  study  we  consider  the  effect  of  priorities 
on  the  design  of  a terrestrial  packet  network.  In  par- 
ticular we  focus  our  attention  on  channel  priorities. 


and  study  the  impact  of  multiple  priority  levels  on  end- 
to-end  delay  bandwidth  performance.  Brief  mention  is 
given  to  path  priorities,  and  it  is  shown  that  path 
priorities  have  only  minor  impact  on  the  performance 
of  a terrestrial  network,  although  they  may  play  a very 
important  role  in  mixed  terrestrial  and  satellite  net- 
works. Access  priorities  are  not  discussed  here,  since 
network  access  control  is  closely  related  to  elaborate 
end-to-end  protocols  and  stability  procedures,  and  there 
fore  would  require  a modeling,  analysis  and  simulation 
effort  far  beyond  the  scope  of  this  study. 

In  the  sequel  some  analytical  results  for  priority 
delay  are  presented,  and  are  applied  to  the  planned 
AUTODIN  II  topological  configuration  and  traffic  environ 
ment . 
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4.3  CHANNEL  PRIORITIES 


4.3.1  Basic  Concepts 

We  consider  a "Head  of  the  Line"  priority  scheme  in  which  a 
high  priority  packet  is  always  transmitted  before  any  low  priority 
packet.  Service  is  first-come  first-served  within  each  priority 
class. 

Preemption  of  partially  transmitted  packets  by  higher  priority 
packets  is  not  considered  since  the  overhead  caused  by  the  retrans- 
mission of  prempted  packets  generally  exceeds  the  marginal  delay  im- 
provement obtained  for  higher  priority  packets  [COLE,  1971] . 

In  our  mathematical  model  we  assume  that  nodal  buffer  storage 
is  infinite,  so  that  there  is  no  overflow.  This  assumption  is  ac- 
ceptable for  the  study  of  network  performance  at  nominal  traffic 
load  levels  and  trunk  utilization  _<  80%,  when  there  are  on  the 
order  of  10  store-and- forward  buffers  available  for  each  output 
line.  However,  if  we  study  the  performance  of  a congested  network, 
and  in  particular  want  to  evaluate  the  effectiveness  of  priorities 
in  preventing  blocking  of  critical  traffic,  we  must  account  for  the 
possibility  of  overflow. 

In  principle,  overflow  will  occur  in  an  overloaded  network  re- 
gardless of  the  size  of  the  store-and-forward  buffer  pool  available. 
The  prowness  of  a network  to  become  overloaded  will  depend  on  the 
flow  control  and  stability  control  implementation.  For  example,  a 
"virtual  circuit"  subnet  is  less  likely  to  become  congested  than  a 
"datagram"  type  net.  In  general  it  is  not  cost-effective,  however, 
to  provide  absolute  protection  against  congestion  using  conservative 
end-to-end  control  schemes,  which  may  reduce  performance  ih  normal 
network  operations. 
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Therefore,  a cost-effective  packet  network  will  experience 
occasional  congestion  and  buffer  overflow  situations.  We  must 
cope  with  such  situations.  In  particular,  to  prevent  deadlocks 
we  may  discard  packets  after  a maximum  number  of  hop  retransmissions. 
To  provide  good  response  time  performance  to  critical  traffic  and 
protect  it  from  blocking,  we  may  create  room  for  it  in  a full  store 
and  forward  node  by  discarding  lower  priority  packets  from  the  queue. 

Returning  to  our  idealized  model  with  infinite  buffer  storage, 
we  recognize  that  such  model  is  adequate  to  study  high  priority 
traffic  performance  in  a congested  network  as  long  as  high  priority 
packets  can  steal  buffers  from  lower  priority  packets  and  high 
priority  traffic  represents  a small  percentage  of  total  network 
traffic.  More  complex  analytical  and  simulation  models  are  re- 
quired to  study  blocking  and  delay  behavior  when  the  above  assump- 
tions are  not  met  [ZEIGLER,  1971] , [LAM,  1975] . 


4.3.2  Delay  Model 

I 

I I 

Given  that  the  traffic  is  divided  into  P channel  priority 
classes  {0,1, .. .P},  where  P has  the  highest  priority,  we  want  to 
obtain  a closed  form  expression  of  average  end-to-end  delay  for 
each  class  as  a function  of  network  load  and  other  network  parameters 
and  traffic  characteristics. 

Let  us  first  recall  that  the  average  queueing  delay  W for 

r 

priority  class  p in  a single  server  system,  with  Poisson  arrival, 
general  service  time  distribution  and  HOL  (Head-of-the-Line)  priority 
discipline  is  given  by  [COBHAM,  1954] : 
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Where : W_  = 


Average  time  to  completion  of  the 
packet  being  currently  transmitted 

= p— - — 

2b 


P 

I p, 

p=o 


= Average  link  utilization 


pp=  Vp 


Xp  = Average  arrival  rate  (Poisson)  of 
class  p customers 


Vip  = Average  class  p service  rate 


b = Total  avg.  service  time  - 


i yvp 


l K 


= Second  moment  of  total  service  time. 


i pk 

k=P 


0<p<P 


p = P+1 
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We  now  consider  a network  with  NN  nodes  and  NA  links  and  cal- 
culate the  average  end-to-end  delay  T for  packets  in  priority 

Am 

class  p.  Applying  Little's  result  to  class  P we  have  [LITTLE,  1961] 


v T = N 
’P  P P 


(4.2) 


where:  yp  = Total  class  p packet  throughput 

[packets/second]. 


Np  = Avg.  number  of  class  p packets 
in  the  network. 


= End-to-end  average  delay. 


Notice  that: 


- I . < 


(4.3) 


Where  N^  is  the  average  number  of  class  p packets  on 
queue  i,  and  NA  is  the  total  number  of  links. 


Applying  Little's  result  to  queue  i we  have: 

N (l)  = X (l)  (W(l)  + tTJl) ) 

P P P P 

Where : 


Avg.  arrival  rate  of  class  p packets 
to  queue  i 


(4.4) 


Average  waiting  time  for  class  p 
at  queue  i. 
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tp^  = Average  service  time  for  class 
p at  queue  i. 


From  Equations  (4.2),  (4.3)  and  (4.4)  we  have: 


. NA 
T = -i-  l 
P Yp  i=l 


XU)  (WU)  + tU)) 


(4.5) 


Recalling  that  the  total  average  end-to-end  delay  T is  de- 
fined as  fellows: 


T = ± ! 


Y T 
*P  P 


where:  Y = I YD 

p=o  H 

we  notice  that  T depends  in  general  on  the  number  and  structure 
of  priority  classes. 

Recalling  further  that  the  total  average  number  of  packets 
in  the  system  N is  given  by: 


N • l N = YT 
p=o  p 

we  observe  that  a priority  strategy  which  minimizes  T will  also 
minimize  the  average  buffer  requirements. 

We  now  apply  the  result  from  Equation  (4.1)  to  the  expression 
of  Tp  in  (4.5)  and  obtain  after  a few  manipulations: 


TP 


if  !i/b  + a.  ^ ♦ d.C,')  (4  • 1 
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Where  s R 


= Total  network  throughput. 


= Traffic  volume  on  link  i (sum 
of  all  priority  classes) . 

= Capacity  of  link  i. 

= Average  packet  length  in  class  p. 
f . 

= Min.  ' {1, 

i 

= Second  moment  of  packet  length 
(over  all  priority  classes) . 


= Average  packet  length  (over  all 


priority  classes) . 

P 

I k.  "y. 

fi  k=P  K K 


, 0 <p  <P 


, p -P+1 


Propagation  and  nodal  processing  delay 
associated  with  the  transmissions  on  link  i. 


In  writing  Equation  (4.4)  we  have  tacitly  assumed  that  the  priority 
traffic  mix  is  the  same  for  all  the  links.  This  will  be  the  case 
if  the  priority  classes  have  all  the  same  point-to-point  traffic  re- 
quirement distribution  and  use  the  same  routes . The  model  however  can 
be  easily  extended  to  handle  requirement  patterns  and  routing  schemes 
which  differ  from  priority  to  priority. 


In  designing  a priority  strategy,  we  wish  to  give  high  prior- 
ity to  critical  traffic,  to  protect  it  from  congestion.  We  also 
wish  to  give  high  priority  to  subnet  control  packets,  to  improve 
network  resource  utilization  and  to  make  control  procedures  (such 
as  routing,  stability  control, etc. ) more  effective.  Finally,  we 
wish  to  assign  high  priority  to  short  packets  to  improve  network 
buffer  utilization  and  reduce  average  packet  delay. 

The  optimal  number  of  priority  classes  is  determined  by  the 
tradeoff  between  delay  performance  and  overhead  introduced  by 
the  implementation  of  each  additional  priority  class.  The  over- 
head can  be  attributed  to  the  following  sources: 


1.  A priority  "tag"  must  be  assigned  to  each 
packet  in  the  Host  or  source  IMP.  This  assign- 
ment may  require  some  processing  (e.g.,  measuring 
packet  length, etc.) 

2.  Each  store-and-forward  node  must  manage  a 
separate  queue  for  each  priority  class. 


3.  If  the  link  control  procedure  imposes  the 
sequential  delivery  of  packets  on  the  link  (e.g., 

HDLC  or  ADCCP) , separate  sequence  numbers  must  be 
maintained  for  each  priority  class  to  allow  higher 
priority  packets  to  "leap  frog"  lower  priority  packets. 

For  most  network  applications  the  optimal  number  of  channel 
priority  classes  is  found  to  range  from  two  to  four.  A larger 
number  of  classes  is  rarely  justified  from  the  point  of  view  of 
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performance  improvement,  while  the  overhead  introduced  may  be  sub- 
stantial [COLE,  1971].  in  this  study  we  consider  only  strategies 
with  2 or  3 priority  classes.  In  particular,  the  following 
strategies  are  considered: 


Priority  1:  Regular  traffic  with  packet  size  <_  L (where 

L is  an  appropriate  threshold) . 


Priority  0:  Regular  traffic  with  packet  size  > L 


First,  we  evaluate  the  delay  performance  for  Strategy  A,  which  is 
the  simplest  strategy  to  implement.  Next,  we  evaluate  the  impact  on 
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network  performance  of  a priority  assignment  separating  control 
traffic  from  critical  traffic  (Strategy  B);  and  short  from  long 
packets  (Strategy  C) . The  comparison  of  the  three  strategies  is 
based  on  a network  application  derived  from  the  proposed  AUTODIN  II 
network  configuration  and  traffic  environment. 
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4.4  AUTODIN  II  APPLICATION 


The  network  under  study  is  one  of  the  proposed  AUTODIN  II  con- 
figurations. Traffic  requirements  correspond  to  the  integration  of 
35  Defense  ADP  Systems  in  a packet  switched  backbone  network.  A 
low  cost  topology  that  accommodates  such  requirements  is  shown  in 
Figure  4.1.  The  details  of  the  topological  optimization  are  de- 
scribed in  [NAC,  1975]. 

The  total  net  traffic  volume  during  the  peak  hour  is  approx- 
imately 1000  Kbps.  To  the  net  traffic  we  add  a 45%  overhead  due 
to  subnet  and  Host-to-Host  protocols.  Further,  we  assume  that 
the  effective  trunk  speed  is  reduced  by  7%  to  account  for  adaptive 
routing  overhead. 

» 

The  topology  displayed  in  Figure  4.1  can  accommodate  the 
above  traffic  requirements  with  an  average  end-to-end  delay  below 
100  msec,  under  the  assumption  that  an  adaptive,  multipath  routing 
algorithm  similar  to  the  one  implemented  in  ARPANET  is  used. 


In  order  to  evaluate  channel  priority  strategies  for  AUTODIN  II, 
we  must  first  introduce  some  assumptions  regarding  the  traffic  char- 
acteristics. * 

We  assume  that  the  critical  (non-blocking)  traffic  amounts  to 
3%  of  the  total  busy  hour  traffic.  This  critical  traffic  volume 
corresponds  to  full  utilization  of  critical  access  channels,  and 
therefore  reflects  the  worst  case  crisis  situation. 


* The  following  assumptions  are  based  in  part  on  ARPANET  ex- 
perience and  in  part  on  reasonable  engineering  judgment.  The  in- 
tent is  not  to  model  the  AUTODIN  II  requirements  accurately,  but 
to  derive  trends  that  can  be  applied  to  networks  with  size,  traffic 
volume,  and  traffic  characteristics  similar  to  AUTODIN  II. 
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We  further  make  the  rather  conservative  assumption  that  each 
packet  requires  a separate  end-to-end  acknowledgement  (similar  to 
the  RFNM  in  ARPANET).  Therefore,  50%  of  the  packets  are  end-to- 
end  ACK's  with  length  assumed  = 152  bits.  These  ACK's  represent 
the  majority  of  the  subnet  control  traffic  (recall  that  routing 
control  traffic  is  accounted  for  by  reducing  effective  trunk  speed) . 
Other  contributions  to  the  subnet  control  traffic  are:  the  mon- 

itoring and  measurement  messages,  the  stability  control  messages, 
and  the  link  control  messages  (non  piggybacked  acknowledgements, 
supervisory  commands , etc .) . To  simplify  the  analysis  we  make  the 
(probably  still  conservative)  assumption  that  50%  of  the  packets 
are  subnet  control  packets  with  length  = 152  bits. 

We  now  study  the  message  and  packet  length  distribution  of 
the  data  traffic  (note:  we  assume  that  both  critical  and  regular 

traffic  have  the  same  distribution).  First,  let  us  assume  that 
the  data  traffic  has  the  following  composition: 


Interactive  and 
Inquiry/Response 


45%  of  total  volume 


Narrative  15%  of  total  volume 

Bulk  40%  of  total  volume 

Interactive  and  Inquiry/Response  messages  correspond  to  man-computer 
and  computer-to-computer  interactions  and  have  average  length  ranging 
from  600  to  6000  bits.  Average  length  for  narrative  and  bulk  traffic 
is  > 10^  bits. 
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Based  on  the  above  message  statistics  and  on  a maximum  pack- 
et . size  = 2000  bits  (plus  168  bits  of  header  and  checksum) , we 
have  the  following  traffic  characteristics: 

Avg.  Packet 


Length 

% Packets 

% Volume 

Subnet  control  traffic 

152 

50 

7.7 

Critical  traffic 

1,800 

1.5 

2.7 

Regular  traffic 

1,800 

48.5 

89.6 

The  packet  length  distribution  for  the  combined  traffic  is  shown 
in  Figure  4.2.  The  average  packet  length  b for  the  combined 
traffic  is: 

b = .972  Kbits 
~2 

The  second  moment  b is: 
b2  = 1.818  Kbits 

The  standard  deviation  a is  given  by: 

(~2  -2  \ 1/2 
a = 1 b2  - b 1 = .93 

Therefore,  the  ratio  a/b  = .95 

Recalling  that  o/b  = 1.  for  the  exponential  distribution  and  that  our 

2 

delay  expressions  depend  solely  on  b and  b (or  a) , as  shown  in 
Equation  (4.6),  we  notice  that  the  actual  packet  distribution  can 
be  reasonably  approximated  with  an  exponential  distribution  of  mean 
b,  in  order  to  calculate  average  delays. 
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According  to  this  traffic  model,  the  total  subnet  protocol 
overhead  (due  to  end-to-end  ACK's  and  header  and  checksum  bits) 
is  approximately  20%.  Considering  our  initial  assumption  of 
total  overhead  = 45%,  this  implies  that  the  remaining  25%  of  over 
head  is  attributable  to  Host-to-Host  protocols.  In  our  model, 
Host  protocol  overhead  is  included  in  the  regular  (or  critical) 
traffic  and  therefore  is  handled  as  regular  (or  critical)  traffic 
accordingly.  In  some  applications,  we  may  assign  higher  priority 
to  Host  protocol  packets,  to  improve  network  bandwidth  perfor- 
mance. We  will  comment  on  this  possibility  in  the  sequel. 

Using  the  above  data  we  now  proceed  to  the  evaluation  of  the 
delay  performance  for  various  priority  schemes. 


The  following  strategies  were  evaluated  for  the  AUTODIN  II 
application : 
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Strategy  C : 

Class  P2:  Critical  traffic  and  subnet  control  traffic. 

Class  P^:  Regular  traffic;  non  full  size  packets. 

{I  < 2168  bits) 

Class  Pq  : Regular  traffic;  full  size  packets  (£,  = 2168  bits). 

Table  4.2  shows  traffic  characteristics  and  performance  for 
the  various  priority  classes  within  each  strategy.  The  perform- 
ance is  measured  in  terms  of  delay  at  nomimal  throughput  level  = 1,000 
Kbps,  assuming  that  optimal  multipath  routing  is  used.  Two  values 
of  delay  are  reported : 

• The  end-to-end  delay,  which  is  the  sum  of  trans- 
mission and  queueing  delays  on  the  average  source- 

destination  path. 

• The  queueing  delay,  which  is  the  sum  of  queueing 

delays  on  the  average  source-destination  path. 

The  queueing  delay  at  a given  load  can  be  easily  calculated  as 
the  difference  between  the  end-to-end  delay  for  such  load  and 
the  end-to-end  delay  at  zero  load. 

Table  4.2  also  reports  the  delay  values  for  the  "non-priority" 
case,  to  allow  comparison  with  the  priority  results. 

Figures  4.3,  4.4  and  4.5  display  delay  vs  throughput  curves 
for  Strategy  A,  B and  C.  End-to-end  delay  for  each  priority  class 
as  well  as  total  average  delay  averaged  over  all  the  classes  are 
calculated  for  various  values  of  throughput,  for  both  single  and 
multipath  routing. 
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Figure  4.6  shows  total  average  delay  for  the  non  priority 
case  and  for  priority  Strategies  A,  B and  C.  Multipath  routing 
is  assumed. 

Examining  the  delay  curves  in  Figures  4.3,  4.4  and  4.5,  the 
preliminary  observation  is  that  multipath  routing  gives  30%  through- 
put performance  improvement  with  respect  to  single  path  routing, 
for  low  priority  traffic.  High  priority  traffic  performance  on  the 
other  hand  is  rather  insensitive  to  the  type  of  routing  discipline 
used.  This  follows  from  the  fact  that  the  relative  link  utilization 
of  high  priority  traffic  is  very  modest,  so  that  queueing  delay 
within  the  high  priority  is  negligible  even  when  the  network  is 
saturated  by  low  priority  traffic.  Transmission  delay  is  there- 
fore the  predominant  component  in  the  high  priority  end-to-end  de- 
lay. Since  transmission  delay  performance  is  optimized  by  single 
path  routing,  better  performance  is  expected  with  single  path  than 
with  multipath  routing  for  high  priority  traffic.  This  conjecture 
is  confirmed  by  the  experimental  results  in  Figures  4.3,  4.4  and 
4.5. 

In  the  sequel  we  assume  that  multipath  routing  is  used  be- 
cause of  the  substantial  throughput  improvement  that  it  provides, 
although  it  may  introduce  a marginal  response  time  degradation  for 
high  priority  traffic. 

Let  us  now  consider  the  impact  of  priorities  on  file  transfer 
performance.  Recall  that  the  effective  file  transfer  bandwidth  is 
given  by  [GERLA,  1974]: 


max} 


B = min  {N 


where  N is  the  number  of  packets  maintained  in  flight  from  source 
to  destination,  b is  the  maximum  packet  size  (-  2168  bits  in  our 
case),  RTD  is  the  round  trip  delay,  sum  of  data  packet  delay  and 
end-to-end  ACK  delay,  and  B is  the  maximum  bandwidth  available 
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from  source  to  destination  through  the  network  using  multiple  paths 
optimized  for  the  given  traffic  load  situation.  From  the  data  in 
Table  4 . 2 and  from  the  curves  in  Figures  4.3,  4.4,  4.5  and  4 . 6 one 
derives  the  following  values  of  RTD,  as  the  sum  of  regular  packet 
delay  and  subnet  control  packet  delay: 
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Type  of  Strategy 

no  priorities 
A & B 
C 

It  then  follows  that,  for  N=1  (only  one  outstanding  packet  allowed) , 
the  introduction  of  priorities  for  the  end-to-end  ACK's  yields  a 
20  to  30%  bandwidth  utilization  improvement  with  respect  to  the 
non  priority  case.  Conversely,  for  N>1  the  priority  implemen- 
tation, reduces  by  20  to  30%  the  number  of  outstanding  packets  (and 
therefore  reassembly  buffers)  required  to  achieve  maximum  bandwidth 
utilization. 

Considering  now  Strategy  B,  we  notice  that  the  difference  be- 
ween  subnet  control  delays  in  Strategies  A and  B is  only  minimal  (see 
Figures  4.3  and  4.4).  We  may  then  conclude  that  it  is  not  necessary 
to  divide  critical  and  subnet  control  traffic  into  two  separate  classes 
in  order  to  optimize  subnet  control  delay  performance.  This  is  true, 
of  course,  only  under  the  condition  that  the  critical  traffic  volume 
is  small  as  compared  to  total  traffic.  If  critical  traffic  were, 
say,  50%  of  the  total  traffic,  two  separate  classes  for  critical  and 
control  traffic  are  required. 

Next,  we  consider  the  Type  C Strategy  and  address  the  problem 
of  selecting  the  optimum  packet  length  4*  to  be  used  as  a threshold 
to  separate  class  from  class  P^.  A value  l*  is  defined  to  be 


RTD 

112  msec 
88 
94 
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optimal  if  the  resulting  priority  class  partition  minimizes  the 
total  average  delay  [COLE,  1971],  In  our  application  Z*  = 2,168 
bits,  i.e.,  regular  full  size  packets  are  assigned  to  class  PQ, 
while  regular  packets  with  l < 2,168  are  assigned  to  P^.  From 
Figure  4.4  we  notice  that  the  introduction  of  the  additional 
priority  class  P.^  provides  a marginal  reduction  of  total  average 
delay  from  43  to  40  msec.  More  important,  the  delay  performance 
of  short  packets  is  substantially  improved  (the  queueing  time  is 
reduced  from  33  msec  to  9 msec) , with  only  minor  degradation  in 
bulk  traffic  delay  (the  queueing  time  is  increased  from  33  msec 
to  39  msec) . The  short  packet  delay  improvement,  although  consider- 
able in  the  relative  sense,  may  not  have  much  practical  impact 
however  on  typical  interactive  applications  characterized  by  short 
packet  traffic,  for  which  a response  time  performance  on  the  order 
of  200  msec  is  generally  considered  satisfactory. 

As  a possible  modification  of  Strategy  C one  may  include  in 
the  intermediate  class  P^  some  traffic  components  which  have  high 
response '•time  requirements,  but  use  full  size  packets  (e.g., 
digitized  speech) . The  performance  should  be  the  same  as  in 
Figure  4.4,  as  long  as  the  volume  of  the  high  response  traffic 
is  small  as  compared  to  total  network  load. 

By  giving  high  priority  to  short  packets  we  also  get,  as  a 
bonus,  a considerable  saving  of  store-and-forward  buffers.  Re- 
call from  Little's  result  [LITTLE,  1961]  that: 

N = yT  (4.7) 

where  N = average  number  of  packets  in  the  network . 

Y = Total  packet  input  rate. 

T = Average  end-to-end  delay. 
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The  average  number  of  store  and  forward  buffers  Q in  use  in  the 
network  is  proportional  to  N (notice  that  Q>N  because  of  the 
copies  of  packets  kept  in  the  output  queues  and  waiting  acknowl- 
edgement) . Thus  from  Equation  (4.7)  Q is  proportional  to  T. 

From  Figure  4.6  we  notice  that  the  average  delay  T at  nominal 
throughput  =*  1000  Kbps  is  reduced  by  30%  (from  55  msec  to  40  msec) 
using  priorities.  This  implies  that  buffer  requirements  are  re- 
duced by  a corresponding  amount. 

Let  us  now  consider  the  impact  of  priorities  on  congestion 
protection.  Congestion  should  be  a rare  event  in  a well-designed 
packet  network.  Flow  control  and  congestion  control  procedures 
maintain  network  performance  within  an  admissible  range  of  values 
by  regulating  input  rates  and  allocating  buffer  resources. 

However,  in  the  design  of  flow  control  and  congestion  control 
procedures  there  is  a tradeoff  between  performance  at  nomimal  load 
and  protection  against  the  worst  case  situation.  It  is  well  rec- 
ognized that  absolute  protection  against  congestion  can  be  achieved 
only  at  the  expense  of  severe  throughput  performance  reduction  for 
nominal  level  operations.  In  the  limit,  absolute  protection  is 
guaranteed  only  by  complete  reservation  of  network  resources  (lines, 
store- and- forward  buffers  and  reassumbly  buffers)  prior  to  trans- 
mission. The  a priori  reservation  limits  the  degree  of  resource 
sharing  in  the  network,  and  consequently  reduces  throughput  per- 
formance . 

Accepting  therefore  the  fact  that  congestion  will  eventually 
occur  even  in  a well-designed  network,  we  must  implement  procedures 
that  permit  delivery  of  subnet  control  packets  (necessary  to  re- 
cover from  congestion)  and  of  critical  traffic  in  case  of  congestion. 
The  channel  priority  implementation  is  the  most  straightforward  of 
such  procedures.  If  we  consider  Figures  4.3,  4.4,  and  4.5,  we  notice 
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that  the  delay  for  critical  and  subnet  control  packets  remains 
low  even  when  the  network  is  completely  saturated  by  lower  priority 
traffic  (note:  values  of  throughput  higher  than  network  capacity 

correspond  to  the  "offered"  input  rate.  Of  the  amount  of  offered 
traffic  in  excess  of  network  capacity,  only  the  critical  and  sub- 
net control  traffic  component  is  accepted  by  the  network.  The  re- 
maining excess  traffic  is  "blocked"  by  input  buffer  overflow) . 
Practically  all  the  priority  strategies  provide  acceptable  pro- 
tection against  congestion.  As  mentioned  in  Section  4.3.1,  we  as- 
sume that  S/F  buffers  are  always  available  for  critical  and  sub- 
net control  traffic.  This  is  obtained  by  discarding  lower  priority 
packets  (as  described  in  Section  4.3.1).  Alternatively,  one  may 
dedicate  output  queue  buffers  to  high  priority  classes. 


..  . 
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4.5  CONCLUSION 

The  implementation  of  a two-level  channel  priority  strategy 
(high  level  = critical  + control  packets;  low  level  = regular 
packets)  in  the  AUTODIN  II  network  gives  substantial  improvements 
with  respect  to  the  non-priority  case.  The  major  performance 
changes  are  summarized  below; 

• 50%  reduction  in  end-to-end  delay  for  high 
priority  packets. 

• 2%  increase  in  end-to-end  delay  for  low 
priority  packets. 

• 20%  reduction  in  average  delay  (total  of 
both  classes) . 

• 20%  bandwidth  improvement  (only  one  out- 
standing packet) . 

• 20%  store-and-forward  buffer  savings. 

• Full  protection  of  high  priority  traffic 
from  blocking  in  case  of  network  congestion. 

Separate  priority  classes  for  subnet  control  traffic  and 
critical  traffic  are  not  justified  in  view  of  the  critical  traffic 
volume  anticipated  for  AUTODIN  II. 


The  introduction  of  an  additional  priority  class  for  short 
packets  (i.e.,  high  priority  = control  + critical  packets;  inter- 
mediate priority  = short  packets;  low  priority  = full  size  packets) 
results  in  a 50%  end-to-end  delay  reduction  for  short  packets. 

This  may  be  beneficial  to  interactive  type  applications  using 
short  packets.  Furthermore,  Host-to-Host  bandwidth  may  be  im- 
proved assuming  that  Host  protocol  messages  are  short  and  there- 
fore, are  assigned  to  this  category  (alternatively,  they  may  be 
assigned  to  the  subnet  control  packet  category) . 

In  summary,  a two-level  channel  priority  structure  is  the  most 
appropriate  for  the  AUTODIN  II  design,  since  it  provides  all  of 
the  benefits  that  can  be  expected  from  a priority  implementation. 

A third  level  of  priority  (for  short  packets,  or  for  interactive 
packets)  is  recommended  only  if  response  time  requirements  are 
very  tight  or  if  Host-to-Host  bandwidth  can  be  significantly  im- 
proved with  a third  priority  class. 
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4.6  DIRECTIONS  FOR  FURTHER  RESEARCH 


Further  research  should  be  directed  to  the  areas  of  path 
priority  and  acceptance  priority  to  obtain  a global  picture  of 
the  impact  and  interaction  of  priorities  on  network  design. 

Path  priorities  are  important  in  networks  with  non-uniform 
propagation  delays  and/or  channel  speeds  (e.g.f  mixed  terrestrial 
and  satellite  networks,  hierarchical  networks , etc. ) . We  need  to 
develop  analytical  models  and  optimization  techniques  to  design 
such  networks  (e.g.,  sizing  of  satellite  and  terrestrial  links) 
to  determine  path  priority  classes  and  to  optimally  allocate 
classes  to  the  alternative  paths.  A parallel  effort  should  be  spent 
in  the  design  of  distributed  adaptive  routing  strategies  for  the 
path  priority  implementation.  Such  routing  strategies  will  be 
similar  in  nature  to  those  developed  for  traditional  packet  net- 
works (e.g.,  ARPANET),  but  will  be  more  elaborate  and  will  take 
into  account  not  only  transmission  and  queueing  delay  estimates, 
but  also  other  performance  estimates  (e.g.,  bandwidth,  hop  dis- 
tance, propagation  delay, etc.). 

Acceptance  priorities  play  a fundamental  role  in  network 
stability  behavior.  In  essence,  network  performance  is  main- 
tained stable  by  the  combined  effort  of  channel  priorities,  access 
priorities,  end-to-end  protocols  and  stability  control  procedures. 
End-to-end  protocols  and  stability  control  procedures  prevent 
global  congestion  by  blocking  the  inputs.  Acceptance  priorities 
provide  adequate  response  time  performance  for  important  traffic 
during  peak  load  situations  (when  regular  traffic  may  be  blocked 
at  the  input  channels) . Channel  priorities  allow  for  delivery  of 
important  traffic  even  when  the  network  is  completely  congested. 

An  important  issue  that  requires  further  investigation  is  the 
optimal  assignment  of  responsibilities  for  the  delivery  of  critical 
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achieve  the  best  utilization  of  network  resources.  For  example, 


if  the  responsibility  relies  entirely  on  access  priorities,  we 
may  need  a rather  conservative  and  "protective"  end-to-end  con- 
trol structure  within  the  subnet  (e.g.,  "virtual  circuit"  struc- 


ture) to  allow  preallocation  of  network  resources.  This  approach 
may  lead  to  inefficient  use  of  such  resources.  If,  on  the  other 
hand,  the  ultimate  responsibility  for  the  delivery  of  non-blocking 
traffic  relies  on  channel  priorities,  we  may  afford  more  permissive 
subnet  control  structures  (e.g.,  "datagram"  structures).  The  latter 
solution,  however,  may  lead  to  longer  trunk  queues,  more  frequent 
store  and  forward  buffer  overflow,  and  more  frequent  loss  of  low 
priority  packets.  The  problem  therefore  consists  of  selecting 
the  optimal  combination  of  subnet  control  procedures,  access 
priorities  and  channel  priorities  to  satisfy  the  performance  re- 
quirements within  each  class  at  least  cost. 

Another  area  that  requires  further  investigation  is  the  dy- 
namic change  of  priority  after  a packet  has  entered  the  network. 

In  this  study  we  have  assumed  that  packet  priority  is  specified 
before  the  packet  is  input  into  the  net.  There  is  some  evidence, 
however,  that  network  performance  may  be  improved  by  increasing, 
for  example,  the  priority  of  packets  that  have  spent  a given  amount 
of  time  in  the  net  (age  priority) , in  order  to  speed  up  their  de- 
livery to  destination  [PICKHOX.TZ,  1974].  Another  example  of  dynam- 
ically changing  priority  structure  consists  of  assigning  min  hop 
path  priority  (i.e.,  guaranteed  delivery  along  the  min  hop  path)  to 
packets  that  have  traveled  a given  number  of  hops,  sey  HN.  Since 
the  min  hop  path  is  always  loop  free,  this  procedure  prevents  packets 
from  remaining  in  a loop  for  more  than  HN  hops. 


Network  Analysis  Corporation 


In  general,  the  change  of  priority  class  during  the  life  of 
a packet  introduces  substantial  overhead  in  the  nodal  processor. 
Therefore,  strategies  with  dynamic  change  of  priority  should  be 
carefully  evaluated  on  the  basis  of  the  tradeoff  between  delay 
performance  and  processor  overhead. 

Finally,  in  this  study  we  have  addressed  the  problem  of  eval- 
uating and  optimizing  average  end-to-end  delays.  In  many  appli- 
cations, however,  we  are  more  interested  in  delay  variance  and 
distribution  rather  than  average  (for  example,  we  may  need  to  eval 
uate  the  delay  which  is  not  exceeded  99%  of  the  time) . Delay 
distributions  are  in  general  difficult  to  evaluate  analytically  in 
computer  networks  because  of  the  correlation  existing  between  the 
various  queues.  Therefore,  some  research  effort  must  be  invested 
in  the  development  of  efficient  simulation  models,  to  study  the 
impact  of  priorities  on  the  delay  distribution  for  various  classes 
of  traffic. 
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